{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Rent Listing Inqueries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 1. 任务描述"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "任务目的： 在Rental Listing Inquiries数据上练习分类方法， 需要根据公寓的特征来预测其受欢迎程度（用户感兴  趣程度分为高、中、低三类）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rental Listing Inquiries数据中，房屋的特征x共有14维，响应值y为用户对该公寓的感兴趣程度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本项目需要：\n",
    "1. 用Logistic回归模型对公寓感兴趣程度进行预测，注意正则超参数的调优。\n",
    "2. 用RBF的SVM对公寓感兴趣程度进行预测，注意正则超参数和核函数参数的调优。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "评价标准为logloss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 导入工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from sklearn.feature_extraction.text import  CountVectorizer\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "\n",
    "from scipy import sparse\n",
    "\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "from sklearn.cluster import KMeans\n",
    "from nltk.metrics import distance as distance\n",
    "\n",
    "#from sklearn.cross_validation import StratifiedKFold\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "from MeanEncoder import MeanEncoder\n",
    "\n",
    "#可视化\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import log_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 数据读取"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#import data\n",
    "dpath = \"../RentListingInqueries/\"\n",
    "train = pd.read_json(dpath + \"RentListingInquries_train.json\")\n",
    "test = pd.read_json(dpath + \"RentListingInquries_test.json\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>building_id</th>\n",
       "      <th>created</th>\n",
       "      <th>description</th>\n",
       "      <th>display_address</th>\n",
       "      <th>features</th>\n",
       "      <th>interest_level</th>\n",
       "      <th>latitude</th>\n",
       "      <th>listing_id</th>\n",
       "      <th>longitude</th>\n",
       "      <th>manager_id</th>\n",
       "      <th>photos</th>\n",
       "      <th>price</th>\n",
       "      <th>street_address</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3</td>\n",
       "      <td>53a5b119ba8f7b61d4e010512e0dfc85</td>\n",
       "      <td>2016-06-24 07:54:24</td>\n",
       "      <td>A Brand New 3 Bedroom 1.5 bath ApartmentEnjoy ...</td>\n",
       "      <td>Metropolitan Avenue</td>\n",
       "      <td>[]</td>\n",
       "      <td>medium</td>\n",
       "      <td>40.7145</td>\n",
       "      <td>7211212</td>\n",
       "      <td>-73.9425</td>\n",
       "      <td>5ba989232d0489da1b5f2c45f6688adc</td>\n",
       "      <td>[https://photos.renthop.com/2/7211212_1ed4542e...</td>\n",
       "      <td>3000</td>\n",
       "      <td>792 Metropolitan Avenue</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10000</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>c5c8a357cba207596b04d1afd1e4f130</td>\n",
       "      <td>2016-06-12 12:19:27</td>\n",
       "      <td></td>\n",
       "      <td>Columbus Avenue</td>\n",
       "      <td>[Doorman, Elevator, Fitness Center, Cats Allow...</td>\n",
       "      <td>low</td>\n",
       "      <td>40.7947</td>\n",
       "      <td>7150865</td>\n",
       "      <td>-73.9667</td>\n",
       "      <td>7533621a882f71e25173b27e3139d83d</td>\n",
       "      <td>[https://photos.renthop.com/2/7150865_be3306c5...</td>\n",
       "      <td>5465</td>\n",
       "      <td>808 Columbus Avenue</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100004</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>c3ba40552e2120b0acfc3cb5730bb2aa</td>\n",
       "      <td>2016-04-17 03:26:41</td>\n",
       "      <td>Top Top West Village location, beautiful Pre-w...</td>\n",
       "      <td>W 13 Street</td>\n",
       "      <td>[Laundry In Building, Dishwasher, Hardwood Flo...</td>\n",
       "      <td>high</td>\n",
       "      <td>40.7388</td>\n",
       "      <td>6887163</td>\n",
       "      <td>-74.0018</td>\n",
       "      <td>d9039c43983f6e564b1482b273bd7b01</td>\n",
       "      <td>[https://photos.renthop.com/2/6887163_de85c427...</td>\n",
       "      <td>2850</td>\n",
       "      <td>241 W 13 Street</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100007</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>28d9ad350afeaab8027513a3e52ac8d5</td>\n",
       "      <td>2016-04-18 02:22:02</td>\n",
       "      <td>Building Amenities - Garage - Garden - fitness...</td>\n",
       "      <td>East 49th Street</td>\n",
       "      <td>[Hardwood Floors, No Fee]</td>\n",
       "      <td>low</td>\n",
       "      <td>40.7539</td>\n",
       "      <td>6888711</td>\n",
       "      <td>-73.9677</td>\n",
       "      <td>1067e078446a7897d2da493d2f741316</td>\n",
       "      <td>[https://photos.renthop.com/2/6888711_6e660cee...</td>\n",
       "      <td>3275</td>\n",
       "      <td>333 East 49th Street</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100013</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>2016-04-28 01:32:41</td>\n",
       "      <td>Beautifully renovated 3 bedroom flex 4 bedroom...</td>\n",
       "      <td>West 143rd Street</td>\n",
       "      <td>[Pre-War]</td>\n",
       "      <td>low</td>\n",
       "      <td>40.8241</td>\n",
       "      <td>6934781</td>\n",
       "      <td>-73.9493</td>\n",
       "      <td>98e13ad4b495b9613cef886d79a6291f</td>\n",
       "      <td>[https://photos.renthop.com/2/6934781_1fa4b41a...</td>\n",
       "      <td>3350</td>\n",
       "      <td>500 West 143rd Street</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        bathrooms  bedrooms                       building_id  \\\n",
       "10            1.5         3  53a5b119ba8f7b61d4e010512e0dfc85   \n",
       "10000         1.0         2  c5c8a357cba207596b04d1afd1e4f130   \n",
       "100004        1.0         1  c3ba40552e2120b0acfc3cb5730bb2aa   \n",
       "100007        1.0         1  28d9ad350afeaab8027513a3e52ac8d5   \n",
       "100013        1.0         4                                 0   \n",
       "\n",
       "                    created  \\\n",
       "10      2016-06-24 07:54:24   \n",
       "10000   2016-06-12 12:19:27   \n",
       "100004  2016-04-17 03:26:41   \n",
       "100007  2016-04-18 02:22:02   \n",
       "100013  2016-04-28 01:32:41   \n",
       "\n",
       "                                              description  \\\n",
       "10      A Brand New 3 Bedroom 1.5 bath ApartmentEnjoy ...   \n",
       "10000                                                       \n",
       "100004  Top Top West Village location, beautiful Pre-w...   \n",
       "100007  Building Amenities - Garage - Garden - fitness...   \n",
       "100013  Beautifully renovated 3 bedroom flex 4 bedroom...   \n",
       "\n",
       "            display_address  \\\n",
       "10      Metropolitan Avenue   \n",
       "10000       Columbus Avenue   \n",
       "100004          W 13 Street   \n",
       "100007     East 49th Street   \n",
       "100013    West 143rd Street   \n",
       "\n",
       "                                                 features interest_level  \\\n",
       "10                                                     []         medium   \n",
       "10000   [Doorman, Elevator, Fitness Center, Cats Allow...            low   \n",
       "100004  [Laundry In Building, Dishwasher, Hardwood Flo...           high   \n",
       "100007                          [Hardwood Floors, No Fee]            low   \n",
       "100013                                          [Pre-War]            low   \n",
       "\n",
       "        latitude  listing_id  longitude                        manager_id  \\\n",
       "10       40.7145     7211212   -73.9425  5ba989232d0489da1b5f2c45f6688adc   \n",
       "10000    40.7947     7150865   -73.9667  7533621a882f71e25173b27e3139d83d   \n",
       "100004   40.7388     6887163   -74.0018  d9039c43983f6e564b1482b273bd7b01   \n",
       "100007   40.7539     6888711   -73.9677  1067e078446a7897d2da493d2f741316   \n",
       "100013   40.8241     6934781   -73.9493  98e13ad4b495b9613cef886d79a6291f   \n",
       "\n",
       "                                                   photos  price  \\\n",
       "10      [https://photos.renthop.com/2/7211212_1ed4542e...   3000   \n",
       "10000   [https://photos.renthop.com/2/7150865_be3306c5...   5465   \n",
       "100004  [https://photos.renthop.com/2/6887163_de85c427...   2850   \n",
       "100007  [https://photos.renthop.com/2/6888711_6e660cee...   3275   \n",
       "100013  [https://photos.renthop.com/2/6934781_1fa4b41a...   3350   \n",
       "\n",
       "                 street_address  \n",
       "10      792 Metropolitan Avenue  \n",
       "10000       808 Columbus Avenue  \n",
       "100004          241 W 13 Street  \n",
       "100007     333 East 49th Street  \n",
       "100013    500 West 143rd Street  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 49352 entries, 10 to 99994\n",
      "Data columns (total 15 columns):\n",
      "bathrooms          49352 non-null float64\n",
      "bedrooms           49352 non-null int64\n",
      "building_id        49352 non-null object\n",
      "created            49352 non-null object\n",
      "description        49352 non-null object\n",
      "display_address    49352 non-null object\n",
      "features           49352 non-null object\n",
      "interest_level     49352 non-null object\n",
      "latitude           49352 non-null float64\n",
      "listing_id         49352 non-null int64\n",
      "longitude          49352 non-null float64\n",
      "manager_id         49352 non-null object\n",
      "photos             49352 non-null object\n",
      "price              49352 non-null int64\n",
      "street_address     49352 non-null object\n",
      "dtypes: float64(3), int64(3), object(9)\n",
      "memory usage: 6.0+ MB\n"
     ]
    }
   ],
   "source": [
    "train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练数据中空值数量为：  0\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Null Count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>street_address</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>price</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>photos</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>manager_id</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>longitude</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>listing_id</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>latitude</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>interest_level</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>features</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>display_address</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>description</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>created</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>building_id</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bedrooms</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bathrooms</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 Null Count\n",
       "street_address            0\n",
       "price                     0\n",
       "photos                    0\n",
       "manager_id                0\n",
       "longitude                 0\n",
       "listing_id                0\n",
       "latitude                  0\n",
       "interest_level            0\n",
       "features                  0\n",
       "display_address           0\n",
       "description               0\n",
       "created                   0\n",
       "building_id               0\n",
       "bedrooms                  0\n",
       "bathrooms                 0"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看训练数据哪些属性有缺失值，并统计数量\n",
    "print(\"训练数据中空值数量为： \", train.isnull().values.sum())\n",
    "train_nulls = pd.DataFrame(train.isnull().sum().sort_values(ascending=False)[:20], columns=['Null Count'])\n",
    "train_nulls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Comment: 训练集中一共有49352条数据，9个非数值类型属性和6个数值类型属性，数据集中无缺失值。\n",
    "也就意味着，对该数据集不用做数据填补工作，但是需要将非数值类型属性值进行编码，使其成为数值类型属性。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 74659 entries, 0 to 99999\n",
      "Data columns (total 14 columns):\n",
      "bathrooms          74659 non-null float64\n",
      "bedrooms           74659 non-null int64\n",
      "building_id        74659 non-null object\n",
      "created            74659 non-null object\n",
      "description        74659 non-null object\n",
      "display_address    74659 non-null object\n",
      "features           74659 non-null object\n",
      "latitude           74659 non-null float64\n",
      "listing_id         74659 non-null int64\n",
      "longitude          74659 non-null float64\n",
      "manager_id         74659 non-null object\n",
      "photos             74659 non-null object\n",
      "price              74659 non-null int64\n",
      "street_address     74659 non-null object\n",
      "dtypes: float64(3), int64(3), object(8)\n",
      "memory usage: 8.5+ MB\n"
     ]
    }
   ],
   "source": [
    "test.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "测试数据中空值数量为：  0\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Null Count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>street_address</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>price</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>photos</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>manager_id</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>longitude</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>listing_id</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>latitude</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>interest_level</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>features</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>display_address</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>description</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>created</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>building_id</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bedrooms</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bathrooms</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 Null Count\n",
       "street_address            0\n",
       "price                     0\n",
       "photos                    0\n",
       "manager_id                0\n",
       "longitude                 0\n",
       "listing_id                0\n",
       "latitude                  0\n",
       "interest_level            0\n",
       "features                  0\n",
       "display_address           0\n",
       "description               0\n",
       "created                   0\n",
       "building_id               0\n",
       "bedrooms                  0\n",
       "bathrooms                 0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看测试数据哪些属性有缺失值，并统计数量\n",
    "print(\"测试数据中空值数量为： \", train.isnull().values.sum())\n",
    "test_nulls = pd.DataFrame(train.isnull().sum().sort_values(ascending=False)[:20], columns=['Null Count'])\n",
    "test_nulls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Comment: 测试集中一共有74659条数据，8个非数值类型属性和6个数值类型属性，数据集中无缺失值。\n",
    "也就意味着，对该数据集也不用做数据填补工作，但是同上，也需要将非数值类型属性值进行编码，使其成为数值类型属性。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>building_id</th>\n",
       "      <th>created</th>\n",
       "      <th>description</th>\n",
       "      <th>display_address</th>\n",
       "      <th>features</th>\n",
       "      <th>latitude</th>\n",
       "      <th>listing_id</th>\n",
       "      <th>longitude</th>\n",
       "      <th>manager_id</th>\n",
       "      <th>photos</th>\n",
       "      <th>price</th>\n",
       "      <th>street_address</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>79780be1514f645d7e6be99a3de696c5</td>\n",
       "      <td>2016-06-11 05:29:41</td>\n",
       "      <td>Large with awesome terrace--accessible via bed...</td>\n",
       "      <td>Suffolk Street</td>\n",
       "      <td>[Elevator, Laundry in Building, Laundry in Uni...</td>\n",
       "      <td>40.7185</td>\n",
       "      <td>7142618</td>\n",
       "      <td>-73.9865</td>\n",
       "      <td>b1b1852c416d78d7765d746cb1b8921f</td>\n",
       "      <td>[https://photos.renthop.com/2/7142618_1c45a2c8...</td>\n",
       "      <td>2950</td>\n",
       "      <td>99 Suffolk Street</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>2016-06-24 06:36:34</td>\n",
       "      <td>Prime Soho - between Bleecker and Houston - Ne...</td>\n",
       "      <td>Thompson Street</td>\n",
       "      <td>[Pre-War, Dogs Allowed, Cats Allowed]</td>\n",
       "      <td>40.7278</td>\n",
       "      <td>7210040</td>\n",
       "      <td>-74.0000</td>\n",
       "      <td>d0b5648017832b2427eeb9956d966a14</td>\n",
       "      <td>[https://photos.renthop.com/2/7210040_d824cc71...</td>\n",
       "      <td>2850</td>\n",
       "      <td>176 Thompson Street</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3dbbb69fd52e0d25131aa1cd459c87eb</td>\n",
       "      <td>2016-06-03 04:29:40</td>\n",
       "      <td>New York chic has reached a new level ...</td>\n",
       "      <td>101 East 10th Street</td>\n",
       "      <td>[Doorman, Elevator, No Fee]</td>\n",
       "      <td>40.7306</td>\n",
       "      <td>7103890</td>\n",
       "      <td>-73.9890</td>\n",
       "      <td>9ca6f3baa475c37a3b3521a394d65467</td>\n",
       "      <td>[https://photos.renthop.com/2/7103890_85b33077...</td>\n",
       "      <td>3758</td>\n",
       "      <td>101 East 10th Street</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>783d21d013a7e655bddc4ed0d461cc5e</td>\n",
       "      <td>2016-06-11 06:17:35</td>\n",
       "      <td>Step into this fantastic new Construction in t...</td>\n",
       "      <td>South Third Street\\r</td>\n",
       "      <td>[Roof Deck, Balcony, Elevator, Laundry in Buil...</td>\n",
       "      <td>40.7109</td>\n",
       "      <td>7143442</td>\n",
       "      <td>-73.9571</td>\n",
       "      <td>0b9d5db96db8472d7aeb67c67338c4d2</td>\n",
       "      <td>[https://photos.renthop.com/2/7143442_0879e9e0...</td>\n",
       "      <td>3300</td>\n",
       "      <td>251  South Third Street\\r</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100000</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2</td>\n",
       "      <td>6134e7c4dd1a98d9aee36623c9872b49</td>\n",
       "      <td>2016-04-12 05:24:17</td>\n",
       "      <td>~Take a stroll in Central Park, enjoy the ente...</td>\n",
       "      <td>Midtown West, 8th Ave</td>\n",
       "      <td>[Common Outdoor Space, Cats Allowed, Dogs Allo...</td>\n",
       "      <td>40.7650</td>\n",
       "      <td>6860601</td>\n",
       "      <td>-73.9845</td>\n",
       "      <td>b5eda0eb31b042ce2124fd9e9fcfce2f</td>\n",
       "      <td>[https://photos.renthop.com/2/6860601_c96164d8...</td>\n",
       "      <td>4900</td>\n",
       "      <td>260 West 54th Street</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        bathrooms  bedrooms                       building_id  \\\n",
       "0             1.0         1  79780be1514f645d7e6be99a3de696c5   \n",
       "1             1.0         2                                 0   \n",
       "100           1.0         1  3dbbb69fd52e0d25131aa1cd459c87eb   \n",
       "1000          1.0         2  783d21d013a7e655bddc4ed0d461cc5e   \n",
       "100000        2.0         2  6134e7c4dd1a98d9aee36623c9872b49   \n",
       "\n",
       "                    created  \\\n",
       "0       2016-06-11 05:29:41   \n",
       "1       2016-06-24 06:36:34   \n",
       "100     2016-06-03 04:29:40   \n",
       "1000    2016-06-11 06:17:35   \n",
       "100000  2016-04-12 05:24:17   \n",
       "\n",
       "                                              description  \\\n",
       "0       Large with awesome terrace--accessible via bed...   \n",
       "1       Prime Soho - between Bleecker and Houston - Ne...   \n",
       "100             New York chic has reached a new level ...   \n",
       "1000    Step into this fantastic new Construction in t...   \n",
       "100000  ~Take a stroll in Central Park, enjoy the ente...   \n",
       "\n",
       "              display_address  \\\n",
       "0              Suffolk Street   \n",
       "1             Thompson Street   \n",
       "100      101 East 10th Street   \n",
       "1000     South Third Street\\r   \n",
       "100000  Midtown West, 8th Ave   \n",
       "\n",
       "                                                 features  latitude  \\\n",
       "0       [Elevator, Laundry in Building, Laundry in Uni...   40.7185   \n",
       "1                   [Pre-War, Dogs Allowed, Cats Allowed]   40.7278   \n",
       "100                           [Doorman, Elevator, No Fee]   40.7306   \n",
       "1000    [Roof Deck, Balcony, Elevator, Laundry in Buil...   40.7109   \n",
       "100000  [Common Outdoor Space, Cats Allowed, Dogs Allo...   40.7650   \n",
       "\n",
       "        listing_id  longitude                        manager_id  \\\n",
       "0          7142618   -73.9865  b1b1852c416d78d7765d746cb1b8921f   \n",
       "1          7210040   -74.0000  d0b5648017832b2427eeb9956d966a14   \n",
       "100        7103890   -73.9890  9ca6f3baa475c37a3b3521a394d65467   \n",
       "1000       7143442   -73.9571  0b9d5db96db8472d7aeb67c67338c4d2   \n",
       "100000     6860601   -73.9845  b5eda0eb31b042ce2124fd9e9fcfce2f   \n",
       "\n",
       "                                                   photos  price  \\\n",
       "0       [https://photos.renthop.com/2/7142618_1c45a2c8...   2950   \n",
       "1       [https://photos.renthop.com/2/7210040_d824cc71...   2850   \n",
       "100     [https://photos.renthop.com/2/7103890_85b33077...   3758   \n",
       "1000    [https://photos.renthop.com/2/7143442_0879e9e0...   3300   \n",
       "100000  [https://photos.renthop.com/2/6860601_c96164d8...   4900   \n",
       "\n",
       "                   street_address  \n",
       "0               99 Suffolk Street  \n",
       "1             176 Thompson Street  \n",
       "100          101 East 10th Street  \n",
       "1000    251  South Third Street\\r  \n",
       "100000       260 West 54th Street  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. 数据探索"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 将非数值类型数据编码"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.1标签interest_level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 将类别型的标签interest_level编码为数字\n",
    "从前面的分析和常识来看，listing_id对确定interest_level没有用，去掉\n",
    "特征编码对训练集和测试集都要做，所以干脆将二者连起来一起处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#将interest_level分为三类，分别给与不同的评分\n",
    "#DataFrame.drop(labels=None, axis=0, index=None, columns=None, level=None, inplace=False, errors='raise')\n",
    "y_map = {'low': 2, 'medium': 1, 'high': 0}\n",
    "train['interest_level'] = train['interest_level'].apply(lambda x: y_map[x])\n",
    "\n",
    "#提取interest_level为y_train，在训练数据中删去listing_id及interest_level列\n",
    "y_train = train.interest_level\n",
    "train = train.drop(['listing_id', 'interest_level'], axis=1)\n",
    "#提取listing_id的值为listing_id，在测试数据中删去listing_id列\n",
    "listing_id = test.listing_id.values\n",
    "test = test.drop('listing_id', axis=1)\n",
    "#获取训练数据的行数\n",
    "ntrain = train.shape[0]\n",
    "#将训练数据和测试数据的属性合并到一起做预处理\n",
    "train_test = pd.concat((train, test), axis=0).reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "y_train = y_train.reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        1\n",
       "1        2\n",
       "2        0\n",
       "3        2\n",
       "4        2\n",
       "5        1\n",
       "6        2\n",
       "7        2\n",
       "8        1\n",
       "9        2\n",
       "10       2\n",
       "11       2\n",
       "12       0\n",
       "13       2\n",
       "14       2\n",
       "15       1\n",
       "16       2\n",
       "17       2\n",
       "18       2\n",
       "19       2\n",
       "20       2\n",
       "21       1\n",
       "22       2\n",
       "23       2\n",
       "24       1\n",
       "25       2\n",
       "26       1\n",
       "27       2\n",
       "28       0\n",
       "29       2\n",
       "        ..\n",
       "49322    2\n",
       "49323    2\n",
       "49324    1\n",
       "49325    1\n",
       "49326    2\n",
       "49327    2\n",
       "49328    2\n",
       "49329    2\n",
       "49330    2\n",
       "49331    2\n",
       "49332    2\n",
       "49333    2\n",
       "49334    2\n",
       "49335    1\n",
       "49336    2\n",
       "49337    1\n",
       "49338    2\n",
       "49339    2\n",
       "49340    2\n",
       "49341    2\n",
       "49342    0\n",
       "49343    2\n",
       "49344    2\n",
       "49345    2\n",
       "49346    1\n",
       "49347    1\n",
       "49348    2\n",
       "49349    2\n",
       "49350    2\n",
       "49351    2\n",
       "Name: interest_level, Length: 49352, dtype: int64"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>building_id</th>\n",
       "      <th>created</th>\n",
       "      <th>description</th>\n",
       "      <th>display_address</th>\n",
       "      <th>features</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>manager_id</th>\n",
       "      <th>photos</th>\n",
       "      <th>price</th>\n",
       "      <th>street_address</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3</td>\n",
       "      <td>53a5b119ba8f7b61d4e010512e0dfc85</td>\n",
       "      <td>2016-06-24 07:54:24</td>\n",
       "      <td>A Brand New 3 Bedroom 1.5 bath ApartmentEnjoy ...</td>\n",
       "      <td>Metropolitan Avenue</td>\n",
       "      <td>[]</td>\n",
       "      <td>40.7145</td>\n",
       "      <td>-73.9425</td>\n",
       "      <td>5ba989232d0489da1b5f2c45f6688adc</td>\n",
       "      <td>[https://photos.renthop.com/2/7211212_1ed4542e...</td>\n",
       "      <td>3000</td>\n",
       "      <td>792 Metropolitan Avenue</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>c5c8a357cba207596b04d1afd1e4f130</td>\n",
       "      <td>2016-06-12 12:19:27</td>\n",
       "      <td></td>\n",
       "      <td>Columbus Avenue</td>\n",
       "      <td>[Doorman, Elevator, Fitness Center, Cats Allow...</td>\n",
       "      <td>40.7947</td>\n",
       "      <td>-73.9667</td>\n",
       "      <td>7533621a882f71e25173b27e3139d83d</td>\n",
       "      <td>[https://photos.renthop.com/2/7150865_be3306c5...</td>\n",
       "      <td>5465</td>\n",
       "      <td>808 Columbus Avenue</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>c3ba40552e2120b0acfc3cb5730bb2aa</td>\n",
       "      <td>2016-04-17 03:26:41</td>\n",
       "      <td>Top Top West Village location, beautiful Pre-w...</td>\n",
       "      <td>W 13 Street</td>\n",
       "      <td>[Laundry In Building, Dishwasher, Hardwood Flo...</td>\n",
       "      <td>40.7388</td>\n",
       "      <td>-74.0018</td>\n",
       "      <td>d9039c43983f6e564b1482b273bd7b01</td>\n",
       "      <td>[https://photos.renthop.com/2/6887163_de85c427...</td>\n",
       "      <td>2850</td>\n",
       "      <td>241 W 13 Street</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>28d9ad350afeaab8027513a3e52ac8d5</td>\n",
       "      <td>2016-04-18 02:22:02</td>\n",
       "      <td>Building Amenities - Garage - Garden - fitness...</td>\n",
       "      <td>East 49th Street</td>\n",
       "      <td>[Hardwood Floors, No Fee]</td>\n",
       "      <td>40.7539</td>\n",
       "      <td>-73.9677</td>\n",
       "      <td>1067e078446a7897d2da493d2f741316</td>\n",
       "      <td>[https://photos.renthop.com/2/6888711_6e660cee...</td>\n",
       "      <td>3275</td>\n",
       "      <td>333 East 49th Street</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>2016-04-28 01:32:41</td>\n",
       "      <td>Beautifully renovated 3 bedroom flex 4 bedroom...</td>\n",
       "      <td>West 143rd Street</td>\n",
       "      <td>[Pre-War]</td>\n",
       "      <td>40.8241</td>\n",
       "      <td>-73.9493</td>\n",
       "      <td>98e13ad4b495b9613cef886d79a6291f</td>\n",
       "      <td>[https://photos.renthop.com/2/6934781_1fa4b41a...</td>\n",
       "      <td>3350</td>\n",
       "      <td>500 West 143rd Street</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms                       building_id              created  \\\n",
       "0        1.5         3  53a5b119ba8f7b61d4e010512e0dfc85  2016-06-24 07:54:24   \n",
       "1        1.0         2  c5c8a357cba207596b04d1afd1e4f130  2016-06-12 12:19:27   \n",
       "2        1.0         1  c3ba40552e2120b0acfc3cb5730bb2aa  2016-04-17 03:26:41   \n",
       "3        1.0         1  28d9ad350afeaab8027513a3e52ac8d5  2016-04-18 02:22:02   \n",
       "4        1.0         4                                 0  2016-04-28 01:32:41   \n",
       "\n",
       "                                         description      display_address  \\\n",
       "0  A Brand New 3 Bedroom 1.5 bath ApartmentEnjoy ...  Metropolitan Avenue   \n",
       "1                                                         Columbus Avenue   \n",
       "2  Top Top West Village location, beautiful Pre-w...          W 13 Street   \n",
       "3  Building Amenities - Garage - Garden - fitness...     East 49th Street   \n",
       "4  Beautifully renovated 3 bedroom flex 4 bedroom...    West 143rd Street   \n",
       "\n",
       "                                            features  latitude  longitude  \\\n",
       "0                                                 []   40.7145   -73.9425   \n",
       "1  [Doorman, Elevator, Fitness Center, Cats Allow...   40.7947   -73.9667   \n",
       "2  [Laundry In Building, Dishwasher, Hardwood Flo...   40.7388   -74.0018   \n",
       "3                          [Hardwood Floors, No Fee]   40.7539   -73.9677   \n",
       "4                                          [Pre-War]   40.8241   -73.9493   \n",
       "\n",
       "                         manager_id  \\\n",
       "0  5ba989232d0489da1b5f2c45f6688adc   \n",
       "1  7533621a882f71e25173b27e3139d83d   \n",
       "2  d9039c43983f6e564b1482b273bd7b01   \n",
       "3  1067e078446a7897d2da493d2f741316   \n",
       "4  98e13ad4b495b9613cef886d79a6291f   \n",
       "\n",
       "                                              photos  price  \\\n",
       "0  [https://photos.renthop.com/2/7211212_1ed4542e...   3000   \n",
       "1  [https://photos.renthop.com/2/7150865_be3306c5...   5465   \n",
       "2  [https://photos.renthop.com/2/6887163_de85c427...   2850   \n",
       "3  [https://photos.renthop.com/2/6888711_6e660cee...   3275   \n",
       "4  [https://photos.renthop.com/2/6934781_1fa4b41a...   3350   \n",
       "\n",
       "            street_address  \n",
       "0  792 Metropolitan Avenue  \n",
       "1      808 Columbus Avenue  \n",
       "2          241 W 13 Street  \n",
       "3     333 East 49th Street  \n",
       "4    500 West 143rd Street  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "124011 , 13\n"
     ]
    }
   ],
   "source": [
    "print(train_test.shape[0], ',', train_test.shape[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.2 price, bathrooms, bedrooms\n",
    "数值型特征，+／-／*／ ／\n",
    "特征的单调变换对XGBoost不必要"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>124011.000000</td>\n",
       "      <td>124011.000000</td>\n",
       "      <td>124011.000000</td>\n",
       "      <td>124011.000000</td>\n",
       "      <td>1.240110e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>1.212622</td>\n",
       "      <td>1.543460</td>\n",
       "      <td>40.737641</td>\n",
       "      <td>-73.949434</td>\n",
       "      <td>3.781324e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.595209</td>\n",
       "      <td>1.110203</td>\n",
       "      <td>0.744337</td>\n",
       "      <td>1.372882</td>\n",
       "      <td>1.582988e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-121.488000</td>\n",
       "      <td>1.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>40.728000</td>\n",
       "      <td>-73.991700</td>\n",
       "      <td>2.495000e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>40.751700</td>\n",
       "      <td>-73.977800</td>\n",
       "      <td>3.150000e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>40.774300</td>\n",
       "      <td>-73.954700</td>\n",
       "      <td>4.100000e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>112.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>44.883500</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>4.490000e+06</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           bathrooms       bedrooms       latitude      longitude  \\\n",
       "count  124011.000000  124011.000000  124011.000000  124011.000000   \n",
       "mean        1.212622       1.543460      40.737641     -73.949434   \n",
       "std         0.595209       1.110203       0.744337       1.372882   \n",
       "min         0.000000       0.000000       0.000000    -121.488000   \n",
       "25%         1.000000       1.000000      40.728000     -73.991700   \n",
       "50%         1.000000       1.000000      40.751700     -73.977800   \n",
       "75%         1.000000       2.000000      40.774300     -73.954700   \n",
       "max       112.000000       8.000000      44.883500       0.000000   \n",
       "\n",
       "              price  \n",
       "count  1.240110e+05  \n",
       "mean   3.781324e+03  \n",
       "std    1.582988e+04  \n",
       "min    1.000000e+00  \n",
       "25%    2.495000e+03  \n",
       "50%    3.150000e+03  \n",
       "75%    4.100000e+03  \n",
       "max    4.490000e+06  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_test.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Comment: 可见房屋均价在3781，而最大值为4490000。我们来看下price的价格分布"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13000.0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ulimit = np.percentile(train_test.price.values, 99)\n",
    "ulimit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对train_test数据中的price去100%的数为13000，对该数据集中大于13000的price赋值为13000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\envs\\python3\\lib\\site-packages\\pandas\\core\\indexing.py:179: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  self._setitem_with_indexer(indexer, value)\n"
     ]
    }
   ],
   "source": [
    "#remove some noise\n",
    "train_test['price'].loc[train_test['price']>13000] = 13000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0         1.5\n",
       "1         1.0\n",
       "2         1.0\n",
       "3         1.0\n",
       "4         1.0\n",
       "5         2.0\n",
       "6         1.0\n",
       "7         2.0\n",
       "8         1.0\n",
       "9         2.0\n",
       "10        1.0\n",
       "11        1.0\n",
       "12        1.0\n",
       "13        2.0\n",
       "14        1.0\n",
       "15        1.0\n",
       "16        1.0\n",
       "17        1.0\n",
       "18        1.0\n",
       "19        1.0\n",
       "20        1.0\n",
       "21        1.0\n",
       "22        1.0\n",
       "23        1.0\n",
       "24        2.0\n",
       "25        3.5\n",
       "26        1.0\n",
       "27        1.0\n",
       "28        1.0\n",
       "29        2.0\n",
       "         ... \n",
       "123981    1.0\n",
       "123982    2.0\n",
       "123983    1.0\n",
       "123984    1.0\n",
       "123985    2.0\n",
       "123986    1.0\n",
       "123987    2.0\n",
       "123988    1.0\n",
       "123989    2.0\n",
       "123990    1.0\n",
       "123991    0.0\n",
       "123992    1.0\n",
       "123993    2.0\n",
       "123994    1.0\n",
       "123995    1.0\n",
       "123996    2.0\n",
       "123997    1.0\n",
       "123998    1.0\n",
       "123999    1.0\n",
       "124000    1.0\n",
       "124001    1.0\n",
       "124002    1.0\n",
       "124003    2.0\n",
       "124004    1.0\n",
       "124005    1.0\n",
       "124006    1.0\n",
       "124007    1.0\n",
       "124008    1.0\n",
       "124009    1.0\n",
       "124010    1.0\n",
       "Name: bathrooms, Length: 124011, dtype: float64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_test.bathrooms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将bathrooms的数据分为三类， 1、1.5、2,其中有奇异值112，把它赋值为1.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# remove some noise\n",
    "train_test.loc[train_test[\"bathrooms\"] == 112, \"bathrooms\"] = 1.5\n",
    "train_test.loc[train_test[\"bathrooms\"] == 10, \"bathrooms\"] = 1\n",
    "train_test.loc[train_test[\"bathrooms\"] == 20, \"bathrooms\"] = 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#构造新特征\n",
    "#room_diff：bathroom房间数 - bedroom房间数\n",
    "#room_num：bathroom房间数 + bedroom房间数\n",
    "train_test[\"room_diff\"] = train_test[\"bathrooms\"] - train_test[\"bedrooms\"]\n",
    "train_test[\"room_num\"] = train_test[\"bedrooms\"] + train_test[\"bathrooms\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.3 Created属性拆分\n",
    "由于created属性里面含有日期，可以把它拆分成数值类型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_test['Date'] = pd.to_datetime(train_test['created'])\n",
    "train_test['Year'] = train_test['Date'].dt.year\n",
    "train_test['Month'] = train_test['Date'].dt.month\n",
    "train_test['Day'] = train_test['Date'].dt.day\n",
    "train_test['Wday'] = train_test['Date'].dt.dayofweek\n",
    "train_test['Yday'] = train_test['Date'].dt.dayofyear\n",
    "train_test['hour'] = train_test['Date'].dt.hour\n",
    "\n",
    "train_test = train_test.drop(['Date', 'created'], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>building_id</th>\n",
       "      <th>description</th>\n",
       "      <th>display_address</th>\n",
       "      <th>features</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>manager_id</th>\n",
       "      <th>photos</th>\n",
       "      <th>price</th>\n",
       "      <th>street_address</th>\n",
       "      <th>room_diff</th>\n",
       "      <th>room_num</th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>Wday</th>\n",
       "      <th>Yday</th>\n",
       "      <th>hour</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3</td>\n",
       "      <td>53a5b119ba8f7b61d4e010512e0dfc85</td>\n",
       "      <td>A Brand New 3 Bedroom 1.5 bath ApartmentEnjoy ...</td>\n",
       "      <td>Metropolitan Avenue</td>\n",
       "      <td>[]</td>\n",
       "      <td>40.7145</td>\n",
       "      <td>-73.9425</td>\n",
       "      <td>5ba989232d0489da1b5f2c45f6688adc</td>\n",
       "      <td>[https://photos.renthop.com/2/7211212_1ed4542e...</td>\n",
       "      <td>3000</td>\n",
       "      <td>792 Metropolitan Avenue</td>\n",
       "      <td>-1.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>24</td>\n",
       "      <td>4</td>\n",
       "      <td>176</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>c5c8a357cba207596b04d1afd1e4f130</td>\n",
       "      <td></td>\n",
       "      <td>Columbus Avenue</td>\n",
       "      <td>[Doorman, Elevator, Fitness Center, Cats Allow...</td>\n",
       "      <td>40.7947</td>\n",
       "      <td>-73.9667</td>\n",
       "      <td>7533621a882f71e25173b27e3139d83d</td>\n",
       "      <td>[https://photos.renthop.com/2/7150865_be3306c5...</td>\n",
       "      <td>5465</td>\n",
       "      <td>808 Columbus Avenue</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>12</td>\n",
       "      <td>6</td>\n",
       "      <td>164</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>c3ba40552e2120b0acfc3cb5730bb2aa</td>\n",
       "      <td>Top Top West Village location, beautiful Pre-w...</td>\n",
       "      <td>W 13 Street</td>\n",
       "      <td>[Laundry In Building, Dishwasher, Hardwood Flo...</td>\n",
       "      <td>40.7388</td>\n",
       "      <td>-74.0018</td>\n",
       "      <td>d9039c43983f6e564b1482b273bd7b01</td>\n",
       "      <td>[https://photos.renthop.com/2/6887163_de85c427...</td>\n",
       "      <td>2850</td>\n",
       "      <td>241 W 13 Street</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>17</td>\n",
       "      <td>6</td>\n",
       "      <td>108</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>28d9ad350afeaab8027513a3e52ac8d5</td>\n",
       "      <td>Building Amenities - Garage - Garden - fitness...</td>\n",
       "      <td>East 49th Street</td>\n",
       "      <td>[Hardwood Floors, No Fee]</td>\n",
       "      <td>40.7539</td>\n",
       "      <td>-73.9677</td>\n",
       "      <td>1067e078446a7897d2da493d2f741316</td>\n",
       "      <td>[https://photos.renthop.com/2/6888711_6e660cee...</td>\n",
       "      <td>3275</td>\n",
       "      <td>333 East 49th Street</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>18</td>\n",
       "      <td>0</td>\n",
       "      <td>109</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>Beautifully renovated 3 bedroom flex 4 bedroom...</td>\n",
       "      <td>West 143rd Street</td>\n",
       "      <td>[Pre-War]</td>\n",
       "      <td>40.8241</td>\n",
       "      <td>-73.9493</td>\n",
       "      <td>98e13ad4b495b9613cef886d79a6291f</td>\n",
       "      <td>[https://photos.renthop.com/2/6934781_1fa4b41a...</td>\n",
       "      <td>3350</td>\n",
       "      <td>500 West 143rd Street</td>\n",
       "      <td>-3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>28</td>\n",
       "      <td>3</td>\n",
       "      <td>119</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms                       building_id  \\\n",
       "0        1.5         3  53a5b119ba8f7b61d4e010512e0dfc85   \n",
       "1        1.0         2  c5c8a357cba207596b04d1afd1e4f130   \n",
       "2        1.0         1  c3ba40552e2120b0acfc3cb5730bb2aa   \n",
       "3        1.0         1  28d9ad350afeaab8027513a3e52ac8d5   \n",
       "4        1.0         4                                 0   \n",
       "\n",
       "                                         description      display_address  \\\n",
       "0  A Brand New 3 Bedroom 1.5 bath ApartmentEnjoy ...  Metropolitan Avenue   \n",
       "1                                                         Columbus Avenue   \n",
       "2  Top Top West Village location, beautiful Pre-w...          W 13 Street   \n",
       "3  Building Amenities - Garage - Garden - fitness...     East 49th Street   \n",
       "4  Beautifully renovated 3 bedroom flex 4 bedroom...    West 143rd Street   \n",
       "\n",
       "                                            features  latitude  longitude  \\\n",
       "0                                                 []   40.7145   -73.9425   \n",
       "1  [Doorman, Elevator, Fitness Center, Cats Allow...   40.7947   -73.9667   \n",
       "2  [Laundry In Building, Dishwasher, Hardwood Flo...   40.7388   -74.0018   \n",
       "3                          [Hardwood Floors, No Fee]   40.7539   -73.9677   \n",
       "4                                          [Pre-War]   40.8241   -73.9493   \n",
       "\n",
       "                         manager_id  \\\n",
       "0  5ba989232d0489da1b5f2c45f6688adc   \n",
       "1  7533621a882f71e25173b27e3139d83d   \n",
       "2  d9039c43983f6e564b1482b273bd7b01   \n",
       "3  1067e078446a7897d2da493d2f741316   \n",
       "4  98e13ad4b495b9613cef886d79a6291f   \n",
       "\n",
       "                                              photos  price  \\\n",
       "0  [https://photos.renthop.com/2/7211212_1ed4542e...   3000   \n",
       "1  [https://photos.renthop.com/2/7150865_be3306c5...   5465   \n",
       "2  [https://photos.renthop.com/2/6887163_de85c427...   2850   \n",
       "3  [https://photos.renthop.com/2/6888711_6e660cee...   3275   \n",
       "4  [https://photos.renthop.com/2/6934781_1fa4b41a...   3350   \n",
       "\n",
       "            street_address  room_diff  room_num  Year  Month  Day  Wday  Yday  \\\n",
       "0  792 Metropolitan Avenue       -1.5       4.5  2016      6   24     4   176   \n",
       "1      808 Columbus Avenue       -1.0       3.0  2016      6   12     6   164   \n",
       "2          241 W 13 Street        0.0       2.0  2016      4   17     6   108   \n",
       "3     333 East 49th Street        0.0       2.0  2016      4   18     0   109   \n",
       "4    500 West 143rd Street       -3.0       5.0  2016      4   28     3   119   \n",
       "\n",
       "   hour  \n",
       "0     7  \n",
       "1    12  \n",
       "2     3  \n",
       "3     2  \n",
       "4     1  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.4 description属性转换数值类型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里把对该房屋的描述转化成字数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# count of words present in description column #\n",
    "train_test[\"num_description_words\"] = train_test[\"description\"].apply(lambda x: len(x.split(\" \")))\n",
    "train_test = train_test.drop(['description'], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>building_id</th>\n",
       "      <th>display_address</th>\n",
       "      <th>features</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>manager_id</th>\n",
       "      <th>photos</th>\n",
       "      <th>price</th>\n",
       "      <th>street_address</th>\n",
       "      <th>room_diff</th>\n",
       "      <th>room_num</th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>Wday</th>\n",
       "      <th>Yday</th>\n",
       "      <th>hour</th>\n",
       "      <th>num_description_words</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3</td>\n",
       "      <td>53a5b119ba8f7b61d4e010512e0dfc85</td>\n",
       "      <td>Metropolitan Avenue</td>\n",
       "      <td>[]</td>\n",
       "      <td>40.7145</td>\n",
       "      <td>-73.9425</td>\n",
       "      <td>5ba989232d0489da1b5f2c45f6688adc</td>\n",
       "      <td>[https://photos.renthop.com/2/7211212_1ed4542e...</td>\n",
       "      <td>3000</td>\n",
       "      <td>792 Metropolitan Avenue</td>\n",
       "      <td>-1.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>24</td>\n",
       "      <td>4</td>\n",
       "      <td>176</td>\n",
       "      <td>7</td>\n",
       "      <td>95</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>c5c8a357cba207596b04d1afd1e4f130</td>\n",
       "      <td>Columbus Avenue</td>\n",
       "      <td>[Doorman, Elevator, Fitness Center, Cats Allow...</td>\n",
       "      <td>40.7947</td>\n",
       "      <td>-73.9667</td>\n",
       "      <td>7533621a882f71e25173b27e3139d83d</td>\n",
       "      <td>[https://photos.renthop.com/2/7150865_be3306c5...</td>\n",
       "      <td>5465</td>\n",
       "      <td>808 Columbus Avenue</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>12</td>\n",
       "      <td>6</td>\n",
       "      <td>164</td>\n",
       "      <td>12</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>c3ba40552e2120b0acfc3cb5730bb2aa</td>\n",
       "      <td>W 13 Street</td>\n",
       "      <td>[Laundry In Building, Dishwasher, Hardwood Flo...</td>\n",
       "      <td>40.7388</td>\n",
       "      <td>-74.0018</td>\n",
       "      <td>d9039c43983f6e564b1482b273bd7b01</td>\n",
       "      <td>[https://photos.renthop.com/2/6887163_de85c427...</td>\n",
       "      <td>2850</td>\n",
       "      <td>241 W 13 Street</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>17</td>\n",
       "      <td>6</td>\n",
       "      <td>108</td>\n",
       "      <td>3</td>\n",
       "      <td>94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>28d9ad350afeaab8027513a3e52ac8d5</td>\n",
       "      <td>East 49th Street</td>\n",
       "      <td>[Hardwood Floors, No Fee]</td>\n",
       "      <td>40.7539</td>\n",
       "      <td>-73.9677</td>\n",
       "      <td>1067e078446a7897d2da493d2f741316</td>\n",
       "      <td>[https://photos.renthop.com/2/6888711_6e660cee...</td>\n",
       "      <td>3275</td>\n",
       "      <td>333 East 49th Street</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>18</td>\n",
       "      <td>0</td>\n",
       "      <td>109</td>\n",
       "      <td>2</td>\n",
       "      <td>80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>West 143rd Street</td>\n",
       "      <td>[Pre-War]</td>\n",
       "      <td>40.8241</td>\n",
       "      <td>-73.9493</td>\n",
       "      <td>98e13ad4b495b9613cef886d79a6291f</td>\n",
       "      <td>[https://photos.renthop.com/2/6934781_1fa4b41a...</td>\n",
       "      <td>3350</td>\n",
       "      <td>500 West 143rd Street</td>\n",
       "      <td>-3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>28</td>\n",
       "      <td>3</td>\n",
       "      <td>119</td>\n",
       "      <td>1</td>\n",
       "      <td>68</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms                       building_id      display_address  \\\n",
       "0        1.5         3  53a5b119ba8f7b61d4e010512e0dfc85  Metropolitan Avenue   \n",
       "1        1.0         2  c5c8a357cba207596b04d1afd1e4f130      Columbus Avenue   \n",
       "2        1.0         1  c3ba40552e2120b0acfc3cb5730bb2aa          W 13 Street   \n",
       "3        1.0         1  28d9ad350afeaab8027513a3e52ac8d5     East 49th Street   \n",
       "4        1.0         4                                 0    West 143rd Street   \n",
       "\n",
       "                                            features  latitude  longitude  \\\n",
       "0                                                 []   40.7145   -73.9425   \n",
       "1  [Doorman, Elevator, Fitness Center, Cats Allow...   40.7947   -73.9667   \n",
       "2  [Laundry In Building, Dishwasher, Hardwood Flo...   40.7388   -74.0018   \n",
       "3                          [Hardwood Floors, No Fee]   40.7539   -73.9677   \n",
       "4                                          [Pre-War]   40.8241   -73.9493   \n",
       "\n",
       "                         manager_id  \\\n",
       "0  5ba989232d0489da1b5f2c45f6688adc   \n",
       "1  7533621a882f71e25173b27e3139d83d   \n",
       "2  d9039c43983f6e564b1482b273bd7b01   \n",
       "3  1067e078446a7897d2da493d2f741316   \n",
       "4  98e13ad4b495b9613cef886d79a6291f   \n",
       "\n",
       "                                              photos  price  \\\n",
       "0  [https://photos.renthop.com/2/7211212_1ed4542e...   3000   \n",
       "1  [https://photos.renthop.com/2/7150865_be3306c5...   5465   \n",
       "2  [https://photos.renthop.com/2/6887163_de85c427...   2850   \n",
       "3  [https://photos.renthop.com/2/6888711_6e660cee...   3275   \n",
       "4  [https://photos.renthop.com/2/6934781_1fa4b41a...   3350   \n",
       "\n",
       "            street_address  room_diff  room_num  Year  Month  Day  Wday  Yday  \\\n",
       "0  792 Metropolitan Avenue       -1.5       4.5  2016      6   24     4   176   \n",
       "1      808 Columbus Avenue       -1.0       3.0  2016      6   12     6   164   \n",
       "2          241 W 13 Street        0.0       2.0  2016      4   17     6   108   \n",
       "3     333 East 49th Street        0.0       2.0  2016      4   18     0   109   \n",
       "4    500 West 143rd Street       -3.0       5.0  2016      4   28     3   119   \n",
       "\n",
       "   hour  num_description_words  \n",
       "0     7                     95  \n",
       "1    12                      9  \n",
       "2     3                     94  \n",
       "3     2                     80  \n",
       "4     1                     68  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.5 manager_id\n",
    "将manager分为几个等级\n",
    "top 1%， 2%， 5， 10， 15， 20， 25， 30， 50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "managers_count = train_test['manager_id'].value_counts()\n",
    "\n",
    "train_test['top_10_manager'] = train_test['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[\n",
    "    managers_count.values >= np.percentile(managers_count.values, 90)] else 0)\n",
    "train_test['top_25_manager'] = train_test['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[\n",
    "    managers_count.values >= np.percentile(managers_count.values, 75)] else 0)\n",
    "train_test['top_5_manager'] = train_test['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[\n",
    "    managers_count.values >= np.percentile(managers_count.values, 95)] else 0)\n",
    "train_test['top_50_manager'] = train_test['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[\n",
    "    managers_count.values >= np.percentile(managers_count.values, 50)] else 0)\n",
    "train_test['top_1_manager'] = train_test['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[\n",
    "    managers_count.values >= np.percentile(managers_count.values, 99)] else 0)\n",
    "train_test['top_2_manager'] = train_test['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[\n",
    "    managers_count.values >= np.percentile(managers_count.values, 98)] else 0)\n",
    "train_test['top_15_manager'] = train_test['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[\n",
    "    managers_count.values >= np.percentile(managers_count.values, 85)] else 0)\n",
    "train_test['top_20_manager'] = train_test['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[\n",
    "    managers_count.values >= np.percentile(managers_count.values, 80)] else 0)\n",
    "train_test['top_30_manager'] = train_test['manager_id'].apply(lambda x: 1 if x in managers_count.index.values[\n",
    "    managers_count.values >= np.percentile(managers_count.values, 70)] else 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 以上代码运行时间3mins"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.6 building_id\n",
    "做法同上"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "buildings_count = train_test['building_id'].value_counts()\n",
    "\n",
    "train_test['top_10_building'] = train_test['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[\n",
    "    buildings_count.values >= np.percentile(buildings_count.values, 90)] else 0)\n",
    "train_test['top_25_building'] = train_test['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[\n",
    "    buildings_count.values >= np.percentile(buildings_count.values, 75)] else 0)\n",
    "train_test['top_5_building'] = train_test['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[\n",
    "    buildings_count.values >= np.percentile(buildings_count.values, 95)] else 0)\n",
    "train_test['top_50_building'] = train_test['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[\n",
    "    buildings_count.values >= np.percentile(buildings_count.values, 50)] else 0)\n",
    "train_test['top_1_building'] = train_test['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[\n",
    "    buildings_count.values >= np.percentile(buildings_count.values, 99)] else 0)\n",
    "train_test['top_2_building'] = train_test['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[\n",
    "    buildings_count.values >= np.percentile(buildings_count.values, 98)] else 0)\n",
    "train_test['top_15_building'] = train_test['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[\n",
    "    buildings_count.values >= np.percentile(buildings_count.values, 85)] else 0)\n",
    "train_test['top_20_building'] = train_test['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[\n",
    "    buildings_count.values >= np.percentile(buildings_count.values, 80)] else 0)\n",
    "train_test['top_30_building'] = train_test['building_id'].apply(lambda x: 1 if x in buildings_count.index.values[\n",
    "    buildings_count.values >= np.percentile(buildings_count.values, 70)] else 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 以上代码运行时间6mins"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.7 photos 转换数值类型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_test['photos_count'] = train_test['photos'].apply(lambda x: len(x))\n",
    "train_test.drop(['photos'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.8 latitude, longtitude 转换数值类型\n",
    "聚类降维编码(#用训练数据训练，对训练数据和测试数据都做变换)\n",
    "到中心的距离（论坛上讨论到曼哈顿中心的距离更好）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    " # Clustering\n",
    "train_location = train_test.loc[:ntrain-1, ['latitude', 'longitude']]\n",
    "test_location = train_test.loc[ntrain:, ['latitude', 'longitude']]\n",
    "\n",
    "kmeans_cluster = KMeans(n_clusters=20)\n",
    "res = kmeans_cluster.fit(train_location)\n",
    "res = kmeans_cluster.predict( pd.concat((train_location, test_location), axis=0).reset_index(drop=True))\n",
    "\n",
    "train_test['cenroid'] = res\n",
    "\n",
    "# L1 distance\n",
    "center = [ train_location['latitude'].mean(), train_location['longitude'].mean()]\n",
    "train_test['distance'] = abs(train_test['latitude'] - center[0]) + abs(train_test['longitude'] - center[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.9 display_address"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_test['display_address'] = train_test['display_address'].apply(lambda x: x.lower().strip())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>building_id</th>\n",
       "      <th>display_address</th>\n",
       "      <th>features</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>manager_id</th>\n",
       "      <th>price</th>\n",
       "      <th>street_address</th>\n",
       "      <th>...</th>\n",
       "      <th>top_5_building</th>\n",
       "      <th>top_50_building</th>\n",
       "      <th>top_1_building</th>\n",
       "      <th>top_2_building</th>\n",
       "      <th>top_15_building</th>\n",
       "      <th>top_20_building</th>\n",
       "      <th>top_30_building</th>\n",
       "      <th>photos_count</th>\n",
       "      <th>cenroid</th>\n",
       "      <th>distance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3</td>\n",
       "      <td>53a5b119ba8f7b61d4e010512e0dfc85</td>\n",
       "      <td>metropolitan avenue</td>\n",
       "      <td>[]</td>\n",
       "      <td>40.7145</td>\n",
       "      <td>-73.9425</td>\n",
       "      <td>5ba989232d0489da1b5f2c45f6688adc</td>\n",
       "      <td>3000</td>\n",
       "      <td>792 Metropolitan Avenue</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>0.040260</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>c5c8a357cba207596b04d1afd1e4f130</td>\n",
       "      <td>columbus avenue</td>\n",
       "      <td>[Doorman, Elevator, Fitness Center, Cats Allow...</td>\n",
       "      <td>40.7947</td>\n",
       "      <td>-73.9667</td>\n",
       "      <td>7533621a882f71e25173b27e3139d83d</td>\n",
       "      <td>5465</td>\n",
       "      <td>808 Columbus Avenue</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>0.064140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>c3ba40552e2120b0acfc3cb5730bb2aa</td>\n",
       "      <td>w 13 street</td>\n",
       "      <td>[Laundry In Building, Dishwasher, Hardwood Flo...</td>\n",
       "      <td>40.7388</td>\n",
       "      <td>-74.0018</td>\n",
       "      <td>d9039c43983f6e564b1482b273bd7b01</td>\n",
       "      <td>2850</td>\n",
       "      <td>241 W 13 Street</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>8</td>\n",
       "      <td>0.048829</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>28d9ad350afeaab8027513a3e52ac8d5</td>\n",
       "      <td>east 49th street</td>\n",
       "      <td>[Hardwood Floors, No Fee]</td>\n",
       "      <td>40.7539</td>\n",
       "      <td>-73.9677</td>\n",
       "      <td>1067e078446a7897d2da493d2f741316</td>\n",
       "      <td>3275</td>\n",
       "      <td>333 East 49th Street</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>19</td>\n",
       "      <td>0.024340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>west 143rd street</td>\n",
       "      <td>[Pre-War]</td>\n",
       "      <td>40.8241</td>\n",
       "      <td>-73.9493</td>\n",
       "      <td>98e13ad4b495b9613cef886d79a6291f</td>\n",
       "      <td>3350</td>\n",
       "      <td>500 West 143rd Street</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>0.088971</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 40 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms                       building_id      display_address  \\\n",
       "0        1.5         3  53a5b119ba8f7b61d4e010512e0dfc85  metropolitan avenue   \n",
       "1        1.0         2  c5c8a357cba207596b04d1afd1e4f130      columbus avenue   \n",
       "2        1.0         1  c3ba40552e2120b0acfc3cb5730bb2aa          w 13 street   \n",
       "3        1.0         1  28d9ad350afeaab8027513a3e52ac8d5     east 49th street   \n",
       "4        1.0         4                                 0    west 143rd street   \n",
       "\n",
       "                                            features  latitude  longitude  \\\n",
       "0                                                 []   40.7145   -73.9425   \n",
       "1  [Doorman, Elevator, Fitness Center, Cats Allow...   40.7947   -73.9667   \n",
       "2  [Laundry In Building, Dishwasher, Hardwood Flo...   40.7388   -74.0018   \n",
       "3                          [Hardwood Floors, No Fee]   40.7539   -73.9677   \n",
       "4                                          [Pre-War]   40.8241   -73.9493   \n",
       "\n",
       "                         manager_id  price           street_address    ...     \\\n",
       "0  5ba989232d0489da1b5f2c45f6688adc   3000  792 Metropolitan Avenue    ...      \n",
       "1  7533621a882f71e25173b27e3139d83d   5465      808 Columbus Avenue    ...      \n",
       "2  d9039c43983f6e564b1482b273bd7b01   2850          241 W 13 Street    ...      \n",
       "3  1067e078446a7897d2da493d2f741316   3275     333 East 49th Street    ...      \n",
       "4  98e13ad4b495b9613cef886d79a6291f   3350    500 West 143rd Street    ...      \n",
       "\n",
       "   top_5_building  top_50_building  top_1_building  top_2_building  \\\n",
       "0               0                1               0               0   \n",
       "1               1                1               0               0   \n",
       "2               1                1               1               1   \n",
       "3               1                1               1               1   \n",
       "4               1                1               1               1   \n",
       "\n",
       "   top_15_building  top_20_building  top_30_building  photos_count  cenroid  \\\n",
       "0                0                0                1             5        7   \n",
       "1                1                1                1            11        0   \n",
       "2                1                1                1             8        8   \n",
       "3                1                1                1             3       19   \n",
       "4                1                1                1             3       11   \n",
       "\n",
       "   distance  \n",
       "0  0.040260  \n",
       "1  0.064140  \n",
       "2  0.048829  \n",
       "3  0.024340  \n",
       "4  0.088971  \n",
       "\n",
       "[5 rows x 40 columns]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1.10 street_address"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_test['street_address'] = train_test['street_address'].apply(lambda x: x.lower().strip())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 类别型特征\n",
    "LabelEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "categoricals = ['building_id', 'manager_id', 'display_address', 'street_address']\n",
    "#categoricals = [x for x in train_test.columns if train_test[x].dtype == 'object']\n",
    "for feat in categoricals:\n",
    "    lbl = LabelEncoder()\n",
    "    lbl.fit(list(train_test[feat].values))\n",
    "    train_test[feat] = lbl.transform(list(train_test[feat].values))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "定义高基数类别型特征编码函数 （manager_id, building_id, display_address,street_address ） 对这些特征进行均值编码（该特征值在每个类别的概率，即原来的一维特征变成了C-1维特征，C为标签类别数目）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Administrator\\RentListing\\MeanEncoder.py:54: FutureWarning: using a dict on a Series for aggregation\n",
      "is deprecated and will be removed in a future version\n",
      "  col_avg_y = X_train.groupby(by=variable, axis=0)['pred_temp'].agg({'mean': 'mean', 'beta': 'size'})\n"
     ]
    }
   ],
   "source": [
    "me = MeanEncoder(categoricals)\n",
    "\n",
    "#trian\n",
    "#import pdb\n",
    "#pdb.set_trace()\n",
    "train_new = train_test.iloc[:ntrain, :]\n",
    "train_new_cat = me.fit_transform(train_new, y_train)\n",
    "\n",
    "#test\n",
    "test_new = train_test.iloc[ntrain:, :]\n",
    "test_new_cat = me.transform(test_new)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_test = pd.concat((train_new_cat, test_new_cat), axis=0).reset_index(drop=True)\n",
    "train_test.drop(categoricals, axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## features\n",
    "描述特征文字长度\n",
    "特征中单词的词频，相##当于以数据集features中出现的词语为字典的one-hot编码（虽然是词频，但在这个任务中每个单词）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_test['features_count'] = train_test['features'].apply(lambda x: len(x))\n",
    "train_test['features2'] = train_test['features']\n",
    "train_test['features2'] = train_test['features2'].apply(lambda x: ' '.join(x))\n",
    "\n",
    "c_vect = CountVectorizer(stop_words='english', max_features=200, ngram_range=(1, 1))\n",
    "c_vect_sparse = c_vect.fit_transform(train_test['features2'])\n",
    "c_vect_sparse_cols = c_vect.get_feature_names()\n",
    "\n",
    "train_test.drop(['features', 'features2'], axis=1, inplace=True)\n",
    "\n",
    "#hstack作为特征处理的最后一部，先将其他所有特征都转换成数值型特征才能处理\n",
    "train_test_sparse = sparse.hstack([train_test, c_vect_sparse]).tocsr()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 特征处理结果存为文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        1\n",
       "1        2\n",
       "2        0\n",
       "3        2\n",
       "4        2\n",
       "5        1\n",
       "6        2\n",
       "7        2\n",
       "8        1\n",
       "9        2\n",
       "10       2\n",
       "11       2\n",
       "12       0\n",
       "13       2\n",
       "14       2\n",
       "15       1\n",
       "16       2\n",
       "17       2\n",
       "18       2\n",
       "19       2\n",
       "20       2\n",
       "21       1\n",
       "22       2\n",
       "23       2\n",
       "24       1\n",
       "25       2\n",
       "26       1\n",
       "27       2\n",
       "28       0\n",
       "29       2\n",
       "        ..\n",
       "49322    2\n",
       "49323    2\n",
       "49324    1\n",
       "49325    1\n",
       "49326    2\n",
       "49327    2\n",
       "49328    2\n",
       "49329    2\n",
       "49330    2\n",
       "49331    2\n",
       "49332    2\n",
       "49333    2\n",
       "49334    2\n",
       "49335    1\n",
       "49336    2\n",
       "49337    1\n",
       "49338    2\n",
       "49339    2\n",
       "49340    2\n",
       "49341    2\n",
       "49342    0\n",
       "49343    2\n",
       "49344    2\n",
       "49345    2\n",
       "49346    1\n",
       "49347    1\n",
       "49348    2\n",
       "49349    2\n",
       "49350    2\n",
       "49351    2\n",
       "Name: interest_level, Length: 49352, dtype: int64"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. 将train_test_new转换成数组\n",
    "2. 将训练集和测试集分离，第49352行为界线\n",
    "3. 将X_train和y_train合并成新的训练集\n",
    "4. 将其写入csv\n",
    "5. 将X_test写入csv文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#存为csv格式方便用excel查看\n",
    "train_test_new = pd.DataFrame(train_test_sparse.toarray())\n",
    "X_train = train_test_new.iloc[:ntrain, :]\n",
    "X_test = train_test_new.iloc[ntrain:, :]\n",
    "\n",
    "train_new = pd.concat((X_train, y_train), axis=1).reset_index(drop=True)\n",
    "train_new.to_csv(dpath + 'RentListingInquries_FE_train.csv', index=False)\n",
    "X_test.to_csv(dpath + 'RentListingInquries_FE_test.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看一下合成之后的训练集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>239</th>\n",
       "      <th>240</th>\n",
       "      <th>241</th>\n",
       "      <th>242</th>\n",
       "      <th>243</th>\n",
       "      <th>244</th>\n",
       "      <th>245</th>\n",
       "      <th>246</th>\n",
       "      <th>247</th>\n",
       "      <th>interest_level</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>40.7145</td>\n",
       "      <td>-73.9425</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>-1.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.7947</td>\n",
       "      <td>-73.9667</td>\n",
       "      <td>5465.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7388</td>\n",
       "      <td>-74.0018</td>\n",
       "      <td>2850.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7539</td>\n",
       "      <td>-73.9677</td>\n",
       "      <td>3275.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>40.8241</td>\n",
       "      <td>-73.9493</td>\n",
       "      <td>3350.0</td>\n",
       "      <td>-3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 249 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     0    1        2        3       4    5    6       7    8     9  \\\n",
       "0  1.5  3.0  40.7145 -73.9425  3000.0 -1.5  4.5  2016.0  6.0  24.0   \n",
       "1  1.0  2.0  40.7947 -73.9667  5465.0 -1.0  3.0  2016.0  6.0  12.0   \n",
       "2  1.0  1.0  40.7388 -74.0018  2850.0  0.0  2.0  2016.0  4.0  17.0   \n",
       "3  1.0  1.0  40.7539 -73.9677  3275.0  0.0  2.0  2016.0  4.0  18.0   \n",
       "4  1.0  4.0  40.8241 -73.9493  3350.0 -3.0  5.0  2016.0  4.0  28.0   \n",
       "\n",
       "        ...        239  240  241  242  243  244  245  246  247  interest_level  \n",
       "0       ...        0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0               1  \n",
       "1       ...        0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0               2  \n",
       "2       ...        0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0               0  \n",
       "3       ...        0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0               2  \n",
       "4       ...        0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0               2  \n",
       "\n",
       "[5 rows x 249 columns]"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_new.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 第一次执行时interest_level那一列好多数据为NaN, 查询了一下api，X_train和y_train对接时是按照对应index对接，但是两者的index都毫无意义所以这里需要做一下处理。以上是y_train经过reset_index之后与X_train合并后的数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##方便下次运行时，直接读取处理过的数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "##X_train = pd.read_csv(dpath + \"RentListingInquries_FE_train.csv\")\n",
    "##X_test = pd.read_csv(dpath + \"RentListingInquries_FE_test.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from  scipy.io import mmwrite\n",
    "\n",
    "X_train_sparse = train_test_sparse[:ntrain, :]\n",
    "X_test_sparse = train_test_sparse[ntrain:, :]\n",
    "\n",
    "train_sparse = sparse.hstack([X_train_sparse, sparse.csr_matrix(y_train).T]).tocsr()\n",
    "\n",
    "mmwrite(dpath + 'RentListingInquries_FE_train.txt',train_sparse)\n",
    "mmwrite(dpath + 'RentListingInquries_FE_test.txt',X_test_sparse)\n",
    "\n",
    "#存为libsvm稀疏格式，直接调用XGBoost的话用稀疏格式更高效\n",
    "#from sklearn.datasets import dump_svmlight_file\n",
    "#dump_svmlight_file(, y_train, dpath + 'RentListingInquries_FE_train.txt',X_train_sparse) \n",
    "#dump_svmlight_file(X_test_sparse,  dpath + 'RentListingInquries_FE_test.txt') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_test_new = pd.DataFrame(train_test_sparse.toarray())\n",
    "X_train = train_test_new.iloc[:ntrain, :]\n",
    "X_test = train_test_new.iloc[ntrain:, :]\n",
    "\n",
    "train_new = pd.concat((X_train, y_train), axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>238</th>\n",
       "      <th>239</th>\n",
       "      <th>240</th>\n",
       "      <th>241</th>\n",
       "      <th>242</th>\n",
       "      <th>243</th>\n",
       "      <th>244</th>\n",
       "      <th>245</th>\n",
       "      <th>246</th>\n",
       "      <th>247</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>40.7145</td>\n",
       "      <td>-73.9425</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>-1.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.7947</td>\n",
       "      <td>-73.9667</td>\n",
       "      <td>5465.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7388</td>\n",
       "      <td>-74.0018</td>\n",
       "      <td>2850.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7539</td>\n",
       "      <td>-73.9677</td>\n",
       "      <td>3275.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>40.8241</td>\n",
       "      <td>-73.9493</td>\n",
       "      <td>3350.0</td>\n",
       "      <td>-3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>40.7429</td>\n",
       "      <td>-74.0028</td>\n",
       "      <td>7995.0</td>\n",
       "      <td>-2.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.8012</td>\n",
       "      <td>-73.9660</td>\n",
       "      <td>3600.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7427</td>\n",
       "      <td>-73.9957</td>\n",
       "      <td>5645.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.8234</td>\n",
       "      <td>-73.9457</td>\n",
       "      <td>1725.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>40.7278</td>\n",
       "      <td>-73.9808</td>\n",
       "      <td>5800.0</td>\n",
       "      <td>-2.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.7769</td>\n",
       "      <td>-73.9467</td>\n",
       "      <td>1950.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.8448</td>\n",
       "      <td>-73.9396</td>\n",
       "      <td>1675.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.7488</td>\n",
       "      <td>-73.9770</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.7707</td>\n",
       "      <td>-73.9817</td>\n",
       "      <td>6895.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7584</td>\n",
       "      <td>-73.9648</td>\n",
       "      <td>3050.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.7439</td>\n",
       "      <td>-73.9743</td>\n",
       "      <td>2350.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.7305</td>\n",
       "      <td>-73.9830</td>\n",
       "      <td>3650.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.8643</td>\n",
       "      <td>-73.9280</td>\n",
       "      <td>1695.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>40.7999</td>\n",
       "      <td>-73.9638</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>-3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7328</td>\n",
       "      <td>-73.9799</td>\n",
       "      <td>3973.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>40.7454</td>\n",
       "      <td>-73.9845</td>\n",
       "      <td>4395.0</td>\n",
       "      <td>-2.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.7427</td>\n",
       "      <td>-73.9794</td>\n",
       "      <td>2999.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7447</td>\n",
       "      <td>-73.9741</td>\n",
       "      <td>2595.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7074</td>\n",
       "      <td>-74.0081</td>\n",
       "      <td>3695.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>2.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>40.7391</td>\n",
       "      <td>-73.9936</td>\n",
       "      <td>7400.0</td>\n",
       "      <td>-2.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>3.5</td>\n",
       "      <td>4.0</td>\n",
       "      <td>40.7584</td>\n",
       "      <td>-73.9653</td>\n",
       "      <td>7500.0</td>\n",
       "      <td>-0.5</td>\n",
       "      <td>7.5</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7728</td>\n",
       "      <td>-73.9502</td>\n",
       "      <td>2295.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7709</td>\n",
       "      <td>-73.9917</td>\n",
       "      <td>3164.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.8335</td>\n",
       "      <td>-73.9141</td>\n",
       "      <td>1350.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.7716</td>\n",
       "      <td>-73.9506</td>\n",
       "      <td>5600.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49322</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7176</td>\n",
       "      <td>-73.9531</td>\n",
       "      <td>3126.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49323</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7655</td>\n",
       "      <td>-73.9785</td>\n",
       "      <td>3325.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49324</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.7721</td>\n",
       "      <td>-73.9539</td>\n",
       "      <td>2100.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49325</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.6182</td>\n",
       "      <td>-74.0360</td>\n",
       "      <td>3500.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49326</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.7829</td>\n",
       "      <td>-73.9731</td>\n",
       "      <td>11950.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49327</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7637</td>\n",
       "      <td>-73.9847</td>\n",
       "      <td>2980.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49328</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7293</td>\n",
       "      <td>-73.9987</td>\n",
       "      <td>2850.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49329</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.7075</td>\n",
       "      <td>-74.0043</td>\n",
       "      <td>2885.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49330</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7287</td>\n",
       "      <td>-73.9808</td>\n",
       "      <td>2950.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49331</th>\n",
       "      <td>1.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.7769</td>\n",
       "      <td>-73.9467</td>\n",
       "      <td>2650.0</td>\n",
       "      <td>1.5</td>\n",
       "      <td>1.5</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49332</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.7957</td>\n",
       "      <td>-73.9705</td>\n",
       "      <td>4850.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49333</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7655</td>\n",
       "      <td>-73.9785</td>\n",
       "      <td>3475.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49334</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.7095</td>\n",
       "      <td>-74.0084</td>\n",
       "      <td>5815.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49335</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7179</td>\n",
       "      <td>-73.9989</td>\n",
       "      <td>2050.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49336</th>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>40.7222</td>\n",
       "      <td>-73.9824</td>\n",
       "      <td>4600.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49337</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.7708</td>\n",
       "      <td>-73.9525</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49338</th>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>40.7766</td>\n",
       "      <td>-73.9463</td>\n",
       "      <td>9200.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49339</th>\n",
       "      <td>1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>40.7695</td>\n",
       "      <td>-73.9603</td>\n",
       "      <td>4550.0</td>\n",
       "      <td>-2.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49340</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.6865</td>\n",
       "      <td>-73.9134</td>\n",
       "      <td>1900.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49341</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.7090</td>\n",
       "      <td>-74.0105</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49342</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7645</td>\n",
       "      <td>-73.9840</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49343</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7728</td>\n",
       "      <td>-73.9591</td>\n",
       "      <td>2675.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49344</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7350</td>\n",
       "      <td>-74.0007</td>\n",
       "      <td>3645.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49345</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7889</td>\n",
       "      <td>-73.9719</td>\n",
       "      <td>2179.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49346</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7296</td>\n",
       "      <td>-73.9869</td>\n",
       "      <td>4500.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49347</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.7426</td>\n",
       "      <td>-73.9790</td>\n",
       "      <td>3200.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49348</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7102</td>\n",
       "      <td>-74.0163</td>\n",
       "      <td>3950.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49349</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>40.7601</td>\n",
       "      <td>-73.9900</td>\n",
       "      <td>2595.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49350</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.7066</td>\n",
       "      <td>-74.0101</td>\n",
       "      <td>3350.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49351</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>40.8699</td>\n",
       "      <td>-73.9172</td>\n",
       "      <td>2200.0</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>49352 rows × 248 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       0    1        2        3        4    5    6       7    8     9   ...   \\\n",
       "0      1.5  3.0  40.7145 -73.9425   3000.0 -1.5  4.5  2016.0  6.0  24.0 ...    \n",
       "1      1.0  2.0  40.7947 -73.9667   5465.0 -1.0  3.0  2016.0  6.0  12.0 ...    \n",
       "2      1.0  1.0  40.7388 -74.0018   2850.0  0.0  2.0  2016.0  4.0  17.0 ...    \n",
       "3      1.0  1.0  40.7539 -73.9677   3275.0  0.0  2.0  2016.0  4.0  18.0 ...    \n",
       "4      1.0  4.0  40.8241 -73.9493   3350.0 -3.0  5.0  2016.0  4.0  28.0 ...    \n",
       "5      2.0  4.0  40.7429 -74.0028   7995.0 -2.0  6.0  2016.0  4.0  19.0 ...    \n",
       "6      1.0  2.0  40.8012 -73.9660   3600.0 -1.0  3.0  2016.0  4.0  27.0 ...    \n",
       "7      2.0  1.0  40.7427 -73.9957   5645.0  1.0  3.0  2016.0  4.0  13.0 ...    \n",
       "8      1.0  1.0  40.8234 -73.9457   1725.0  0.0  2.0  2016.0  4.0  20.0 ...    \n",
       "9      2.0  4.0  40.7278 -73.9808   5800.0 -2.0  6.0  2016.0  4.0   2.0 ...    \n",
       "10     1.0  0.0  40.7769 -73.9467   1950.0  1.0  1.0  2016.0  4.0  14.0 ...    \n",
       "11     1.0  1.0  40.8448 -73.9396   1675.0  0.0  2.0  2016.0  6.0   3.0 ...    \n",
       "12     1.0  2.0  40.7488 -73.9770   3000.0 -1.0  3.0  2016.0  4.0  19.0 ...    \n",
       "13     2.0  2.0  40.7707 -73.9817   6895.0  0.0  4.0  2016.0  4.0   9.0 ...    \n",
       "14     1.0  1.0  40.7584 -73.9648   3050.0  0.0  2.0  2016.0  6.0   1.0 ...    \n",
       "15     1.0  0.0  40.7439 -73.9743   2350.0  1.0  1.0  2016.0  4.0  18.0 ...    \n",
       "16     1.0  2.0  40.7305 -73.9830   3650.0 -1.0  3.0  2016.0  4.0  22.0 ...    \n",
       "17     1.0  1.0  40.8643 -73.9280   1695.0  0.0  2.0  2016.0  4.0  19.0 ...    \n",
       "18     1.0  4.0  40.7999 -73.9638   5000.0 -3.0  5.0  2016.0  4.0  20.0 ...    \n",
       "19     1.0  1.0  40.7328 -73.9799   3973.0  0.0  2.0  2016.0  4.0   9.0 ...    \n",
       "20     1.0  3.0  40.7454 -73.9845   4395.0 -2.0  4.0  2016.0  4.0  12.0 ...    \n",
       "21     1.0  2.0  40.7427 -73.9794   2999.0 -1.0  3.0  2016.0  4.0   7.0 ...    \n",
       "22     1.0  1.0  40.7447 -73.9741   2595.0  0.0  2.0  2016.0  4.0  13.0 ...    \n",
       "23     1.0  1.0  40.7074 -74.0081   3695.0  0.0  2.0  2016.0  4.0  17.0 ...    \n",
       "24     2.0  4.0  40.7391 -73.9936   7400.0 -2.0  6.0  2016.0  6.0   7.0 ...    \n",
       "25     3.5  4.0  40.7584 -73.9653   7500.0 -0.5  7.5  2016.0  4.0  27.0 ...    \n",
       "26     1.0  1.0  40.7728 -73.9502   2295.0  0.0  2.0  2016.0  4.0  25.0 ...    \n",
       "27     1.0  1.0  40.7709 -73.9917   3164.0  0.0  2.0  2016.0  4.0  22.0 ...    \n",
       "28     1.0  1.0  40.8335 -73.9141   1350.0  0.0  2.0  2016.0  4.0  21.0 ...    \n",
       "29     2.0  2.0  40.7716 -73.9506   5600.0  0.0  4.0  2016.0  4.0  26.0 ...    \n",
       "...    ...  ...      ...      ...      ...  ...  ...     ...  ...   ... ...    \n",
       "49322  1.0  1.0  40.7176 -73.9531   3126.0  0.0  2.0  2016.0  4.0  25.0 ...    \n",
       "49323  1.0  1.0  40.7655 -73.9785   3325.0  0.0  2.0  2016.0  4.0  26.0 ...    \n",
       "49324  1.0  0.0  40.7721 -73.9539   2100.0  1.0  1.0  2016.0  4.0   8.0 ...    \n",
       "49325  1.0  2.0  40.6182 -74.0360   3500.0 -1.0  3.0  2016.0  4.0  29.0 ...    \n",
       "49326  2.0  2.0  40.7829 -73.9731  11950.0  0.0  4.0  2016.0  4.0   5.0 ...    \n",
       "49327  1.0  1.0  40.7637 -73.9847   2980.0  0.0  2.0  2016.0  4.0  24.0 ...    \n",
       "49328  1.0  1.0  40.7293 -73.9987   2850.0  0.0  2.0  2016.0  4.0  25.0 ...    \n",
       "49329  1.0  2.0  40.7075 -74.0043   2885.0 -1.0  3.0  2016.0  4.0   3.0 ...    \n",
       "49330  1.0  1.0  40.7287 -73.9808   2950.0  0.0  2.0  2016.0  4.0  19.0 ...    \n",
       "49331  1.5  0.0  40.7769 -73.9467   2650.0  1.5  1.5  2016.0  4.0  14.0 ...    \n",
       "49332  1.0  2.0  40.7957 -73.9705   4850.0 -1.0  3.0  2016.0  6.0   6.0 ...    \n",
       "49333  1.0  1.0  40.7655 -73.9785   3475.0  0.0  2.0  2016.0  4.0  26.0 ...    \n",
       "49334  2.0  2.0  40.7095 -74.0084   5815.0  0.0  4.0  2016.0  4.0  16.0 ...    \n",
       "49335  1.0  1.0  40.7179 -73.9989   2050.0  0.0  2.0  2016.0  4.0   5.0 ...    \n",
       "49336  2.0  3.0  40.7222 -73.9824   4600.0 -1.0  5.0  2016.0  4.0  22.0 ...    \n",
       "49337  1.0  0.0  40.7708 -73.9525   2000.0  1.0  1.0  2016.0  4.0  29.0 ...    \n",
       "49338  2.0  3.0  40.7766 -73.9463   9200.0 -1.0  5.0  2016.0  4.0   3.0 ...    \n",
       "49339  1.0  3.0  40.7695 -73.9603   4550.0 -2.0  4.0  2016.0  4.0  15.0 ...    \n",
       "49340  1.0  1.0  40.6865 -73.9134   1900.0  0.0  2.0  2016.0  4.0  14.0 ...    \n",
       "49341  1.0  0.0  40.7090 -74.0105   2500.0  1.0  1.0  2016.0  4.0  27.0 ...    \n",
       "49342  1.0  1.0  40.7645 -73.9840   2500.0  0.0  2.0  2016.0  4.0   7.0 ...    \n",
       "49343  1.0  1.0  40.7728 -73.9591   2675.0  0.0  2.0  2016.0  4.0   2.0 ...    \n",
       "49344  1.0  1.0  40.7350 -74.0007   3645.0  0.0  2.0  2016.0  4.0  14.0 ...    \n",
       "49345  1.0  1.0  40.7889 -73.9719   2179.0  0.0  2.0  2016.0  4.0  11.0 ...    \n",
       "49346  1.0  1.0  40.7296 -73.9869   4500.0  0.0  2.0  2016.0  4.0  22.0 ...    \n",
       "49347  1.0  2.0  40.7426 -73.9790   3200.0 -1.0  3.0  2016.0  6.0   2.0 ...    \n",
       "49348  1.0  1.0  40.7102 -74.0163   3950.0  0.0  2.0  2016.0  4.0   4.0 ...    \n",
       "49349  1.0  1.0  40.7601 -73.9900   2595.0  0.0  2.0  2016.0  4.0  16.0 ...    \n",
       "49350  1.0  0.0  40.7066 -74.0101   3350.0  1.0  1.0  2016.0  4.0   8.0 ...    \n",
       "49351  1.0  2.0  40.8699 -73.9172   2200.0 -1.0  3.0  2016.0  4.0  12.0 ...    \n",
       "\n",
       "       238  239  240  241  242  243  244  245  246  247  \n",
       "0      0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "1      0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "2      0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "3      0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "4      0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "5      0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "6      0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "7      0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "8      0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "9      0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "10     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "11     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "12     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "13     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "14     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "15     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "16     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "17     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "18     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "19     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "20     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "21     0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "22     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "23     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "24     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "25     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "26     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "27     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "28     0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "29     0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "...    ...  ...  ...  ...  ...  ...  ...  ...  ...  ...  \n",
       "49322  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49323  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49324  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49325  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49326  0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49327  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49328  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49329  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49330  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49331  0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49332  0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49333  0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49334  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49335  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49336  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49337  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49338  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49339  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49340  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49341  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49342  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49343  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49344  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49345  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49346  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49347  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49348  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49349  0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49350  0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "49351  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  \n",
       "\n",
       "[49352 rows x 248 columns]"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        1\n",
       "1        2\n",
       "2        0\n",
       "3        2\n",
       "4        2\n",
       "5        1\n",
       "6        2\n",
       "7        2\n",
       "8        1\n",
       "9        2\n",
       "10       2\n",
       "11       2\n",
       "12       0\n",
       "13       2\n",
       "14       2\n",
       "15       1\n",
       "16       2\n",
       "17       2\n",
       "18       2\n",
       "19       2\n",
       "20       2\n",
       "21       1\n",
       "22       2\n",
       "23       2\n",
       "24       1\n",
       "25       2\n",
       "26       1\n",
       "27       2\n",
       "28       0\n",
       "29       2\n",
       "        ..\n",
       "49322    2\n",
       "49323    2\n",
       "49324    1\n",
       "49325    1\n",
       "49326    2\n",
       "49327    2\n",
       "49328    2\n",
       "49329    2\n",
       "49330    2\n",
       "49331    2\n",
       "49332    2\n",
       "49333    2\n",
       "49334    2\n",
       "49335    1\n",
       "49336    2\n",
       "49337    1\n",
       "49338    2\n",
       "49339    2\n",
       "49340    2\n",
       "49341    2\n",
       "49342    0\n",
       "49343    2\n",
       "49344    2\n",
       "49345    2\n",
       "49346    1\n",
       "49347    1\n",
       "49348    2\n",
       "49349    2\n",
       "49350    2\n",
       "49351    2\n",
       "Name: interest_level, Length: 49352, dtype: int64"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(49352, 249)"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_new.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 现在新的训练集看起来就正常多了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据预处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#数据标准化\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "#初始化特征的标准化器\n",
    "ss_X = StandardScaler()\n",
    "\n",
    "#分别对训练和测试数据的特征进行标准化处理\n",
    "X_train = ss_X.fit_transform(X_train)\n",
    "X_test = ss_X.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### default Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "lr = LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\envs\\python3\\lib\\site-packages\\sklearn\\cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
      "  \"This module will be removed in 0.20.\", DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logloss of each fold is:  [ 0.68026871  0.67515314  0.66678082  0.65855088  0.67339909]\n",
      "cv logloss is:  0.670830528213\n"
     ]
    }
   ],
   "source": [
    "#交叉验证用于评估模型性能和进行参数调优（模型选择）\n",
    "#分类任务中交叉验证缺省是采用StratifiedKFold\n",
    "from sklearn.cross_validation import cross_val_score\n",
    "loss = cross_val_score(lr, X_train, y_train, cv=5, scoring='neg_log_loss')\n",
    "print('logloss of each fold is: ', -loss)\n",
    "print('cv logloss is: ', -loss.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 以上代码运行时间6-7mins"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 正则化的参数调优"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "logistic回归需要调整超参数有：\n",
    "C -- 正则系数，一般在log域（取log后的值），均匀设置候选参数\n",
    "penalty -- L2/L1\n",
    "目标函数： J = sum（logloss(f(xi), yi) + C * penalty\n",
    "在sklearn框架下，不同学习器的参数调整步骤相同：\n",
    "设置候选参数集合\n",
    "调用GridSearchCV\n",
    "调用fit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'penalty': ['l1', 'l2'], 'C': [0.01, 0.1, 1]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score=True,\n",
       "       scoring='neg_log_loss', verbose=0)"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "#需要调优的参数\n",
    "\n",
    "penaltys = [\"l1\", \"l2\"]\n",
    "Cs = [0.01, 0.1, 1]\n",
    "tuned_parameters = dict(penalty = penaltys, C = Cs)\n",
    "\n",
    "lr_penalty = LogisticRegression()\n",
    "grid = GridSearchCV(lr_penalty, tuned_parameters, cv=5, scoring='neg_log_loss')\n",
    "grid.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4hours"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([   9.99371338,   14.22076139,   96.01130152,   31.72621469,\n",
       "         336.00401816,   72.45414414]),\n",
       " 'mean_score_time': array([ 0.02544122,  0.02032094,  0.02500148,  0.01680088,  0.02380152,\n",
       "         0.01800089]),\n",
       " 'mean_test_score': array([-0.66621808, -0.66880039, -0.66478293, -0.66877419, -0.66825197,\n",
       "        -0.67083037]),\n",
       " 'mean_train_score': array([-0.66212555, -0.65572455, -0.65192267, -0.65086983, -0.64977107,\n",
       "        -0.64982827]),\n",
       " 'param_C': masked_array(data = [0.01 0.01 0.1 0.1 1 1],\n",
       "              mask = [False False False False False False],\n",
       "        fill_value = ?),\n",
       " 'param_penalty': masked_array(data = ['l1' 'l2' 'l1' 'l2' 'l1' 'l2'],\n",
       "              mask = [False False False False False False],\n",
       "        fill_value = ?),\n",
       " 'params': [{'C': 0.01, 'penalty': 'l1'},\n",
       "  {'C': 0.01, 'penalty': 'l2'},\n",
       "  {'C': 0.1, 'penalty': 'l1'},\n",
       "  {'C': 0.1, 'penalty': 'l2'},\n",
       "  {'C': 1, 'penalty': 'l1'},\n",
       "  {'C': 1, 'penalty': 'l2'}],\n",
       " 'rank_test_score': array([2, 5, 1, 4, 3, 6]),\n",
       " 'split0_test_score': array([-0.67599563, -0.67792373, -0.67484196, -0.67773345, -0.67807288,\n",
       "        -0.68026871]),\n",
       " 'split0_train_score': array([-0.65919719, -0.65327383, -0.64945191, -0.64847733, -0.64759082,\n",
       "        -0.64764039]),\n",
       " 'split1_test_score': array([-0.66591237, -0.6701723 , -0.66494629, -0.67094617, -0.67192526,\n",
       "        -0.67515314]),\n",
       " 'split1_train_score': array([-0.66233448, -0.65565134, -0.65203159, -0.65082298, -0.64998865,\n",
       "        -0.64997674]),\n",
       " 'split2_test_score': array([-0.66350037, -0.66758126, -0.66447884, -0.66826297, -0.66454861,\n",
       "        -0.66678082]),\n",
       " 'split2_train_score': array([-0.66287981, -0.65628346, -0.65202079, -0.65111511, -0.64891915,\n",
       "        -0.64917848]),\n",
       " 'split3_test_score': array([-0.65724023, -0.65768064, -0.65173824, -0.6558287 , -0.65573141,\n",
       "        -0.65855088]),\n",
       " 'split3_train_score': array([-0.66517518, -0.65855735, -0.65508353, -0.65394439, -0.65319389,\n",
       "        -0.65318369]),\n",
       " 'split4_test_score': array([-0.66844245, -0.67064457, -0.66791026, -0.6711004 , -0.67098251,\n",
       "        -0.67339909]),\n",
       " 'split4_train_score': array([-0.66104107, -0.65485679, -0.65102555, -0.64998932, -0.64916283,\n",
       "        -0.64916207]),\n",
       " 'std_fit_time': array([  0.64235218,   1.01401977,  20.41207277,   0.93749706,\n",
       "         20.94060386,   5.39899265]),\n",
       " 'std_score_time': array([ 0.00333054,  0.00337224,  0.00282859,  0.00040007,  0.00271317,\n",
       "         0.00167337]),\n",
       " 'std_test_score': array([ 0.00614231,  0.00653706,  0.00750015,  0.00718691,  0.00758807,\n",
       "         0.00750422]),\n",
       " 'std_train_score': array([ 0.00198314,  0.0017379 ,  0.00183967,  0.00178967,  0.00187682,\n",
       "         0.00184031])}"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.cv_results_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.664782926508\n",
      "{'C': 0.1, 'penalty': 'l1'}\n"
     ]
    }
   ],
   "source": [
    "# Examine the best model\n",
    "print(-grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 由于运行时间受限，缩小了，参数的选取范围，故目前得到最优参数为0.1, L1损失，评分为0.66, 与缺省方法得到的结果（0.67）相比，损失变小了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# plot CV误差曲线\n",
    "test_means = grid.cv_results_[\"mean_test_score\"]\n",
    "test_stds = grid.cv_results_[\"std_test_score\"]\n",
    "train_means = grid.cv_results_[\"mean_train_score\"]\n",
    "train_stds = grid.cv_results_[\"std_train_score\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.66621808, -0.66880039, -0.66478293, -0.66877419, -0.66825197,\n",
       "       -0.67083037])"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.array(test_means)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VPW98PHPd7JNNiAbCARMwKBAwlIRtLhrlap1F9Gi\n1mppvfrc1j71qbZ9vC5d9PFa23tr60KL6y22tii11l2rpUVBZHdDNsNOSFiyL9/nj99JmCSTZJKZ\nyUyS7/v1mhdnzvmdM9+ZDPnmtx5RVYwxxpie8sU6AGOMMX2bJRJjjDFhsURijDEmLJZIjDHGhMUS\niTHGmLBYIjHGGBMWSyTGGGPCYonEGGNMWCyRGGOMCUtirAPoDbm5uVpQUBDrMIwxpk95//3396pq\nXlflBkQiKSgoYPny5bEOwxhj+hQR2RJKOWvaMsYYExZLJMYYY8JiicQYY0xYBkQfSTD19fWUlpZS\nU1MT61Dilt/vJz8/n6SkpFiHYoyJYwM2kZSWlpKZmUlBQQEiEutw4o6qUlZWRmlpKYWFhbEOxxgT\nxwZs01ZNTQ05OTmWRDogIuTk5FiNzRjTpQGbSABLIl2wz8cYE4oBnUi66/KH/8XlD/8r1mEYY0xc\nsUQSQxkZGS3bs2bNYsiQIZx33nlBy954441MmTKFCRMmkJqaypQpU5gyZQrPPvtst15zxYoVvPTS\nS2HFbYyJf9e+dC3XvnRtr7zWgO1sjze33HILVVVVPPzww0GPP/jggwBs3ryZ8847j5UrV/bodVas\nWMHatWuZNWtWj2M1xphAViOJE2eccQaZmZk9OvfTTz/l7LPP5thjj+Xkk0/mk08+AWDhwoUUFxcz\nefJkTjvtNKqrq7nrrrt4+umne1SbMcaYYKxGAtz5l3Ws336gy3Lrd7gyofSTTBgxiP/4ysSwYwvF\nvHnzmD9/PmPHjmXJkiXcdNNNvPLKK9x555289dZbDBs2jIqKClJTU7n99ttZu3Ytv/jFL3olNmNM\n/2eJpI+rqKhg6dKlXHLJJS37GhoaAJg5cyZXX301l112GRdffHGsQjTG9HOWSCDkmkNzTeSZb54Q\nzXC6RVXJzc0N2mfy6KOP8u677/LCCy8wefJkVq9eHYMIjTH9nfWR9HFZWVkMHz6cRYsWAdDU1MSq\nVasA2LhxI8cffzx33303WVlZbNu2jczMTA4ePBjLkI0x/Ywlkjhx0kkncdlll/H666+Tn5/Pyy+/\nHPK5Cxcu5KGHHmLy5MlMnDiRF154AYCbb76ZkpISSkpKOPPMMykuLub0009n1apVTJ061TrbjTER\nYU1bMXTo0KGW7XfeeSekcwoKCli7dm2rfWPGjAmaeBYvXtxuX15ent3kyxgTUZZIuiGe+kaMMSZe\nWNOWMcaYsFgiMcYYExZLJMYYY8JiicQYY0xYLJF0x4Jz3cMYY0wLSyQx1NvLyC9atIj77rsv7LiN\nMSaQDf+NE5FaRr6hoYHExOA/1osuuigywZqIaL5XxIJZC2IciemPmheZ7Q1WI4kT4Swjf+KJJ/LD\nH/6Qk08+mV/96lc8//zzzJgxg6lTp3LWWWexe/duAObPn893vvMdAObOncu3v/1tvvjFLzJmzJiW\nJVaMMaa7olojEZFZwC+BBGC+qt4TpMxs4A5AgVWqeqWInAY8EFDsGGCOqj4nIo8BpwD7vWNfU9We\n3eWp2d9uhZ1rui6301v0MJR+kiNK4Mvt3m7UHDhwgLfffhuA8vJyzj//fESEhx56iPvvv5977723\n3Tm7d+9myZIlrFmzhtmzZ1uNxRjTI1FLJCKSADwIfAkoBZaJyGJVXR9Qpgi4DZipquUiMhRAVd8E\npnhlsoENwCsBl79FVW2hqABz5sxp2d66dSuzZ89m586d1NbWMm7cuKDnXHjhhYgIkyZNYtu2bb0V\nqjGmn4lmjWQ6sEFVNwKIyELgAmB9QJlvAA+qajmAqu4Ocp1Lgb+palXUIu2g5rBp/yYACgcXuh3N\nNZFr/xq1UHoqPT29ZfvGG2/kBz/4Aeeccw6vvfYa99wT/P2lpKS0bKtq1GM0xvRP0ewjGQl8HvC8\n1NsXaBwwTkSWiMhSrymsrTnA79vs+4mIrBaRB0QkJcg5A9r+/fsZOXIkqsrjjz8e63CMMf1cNBOJ\nBNnX9s/eRKAIOBW4ApgvIkNaLiAyHCgBApe2vQ3XZ3IckA18P+iLi8wTkeUisnzPnj09fQ+9Jpxl\n5Nu64447uOiiizjllFMYNmxYBKM0xpj2otm0VQqMCnieD2wPUmapqtYDm0TkY1xiWeYdnw0s8o4D\noKo7vM1aEVkAfC/Yi6vqI8AjANOmTYvLdptILSP/j3/8o9XzSy65pNWtd5tdf/31LdtPPfVUh7EY\nY0x3RLNGsgwoEpFCEUnGNVG1vUHGc8BpACKSi2vq2hhw/AraNGt5tRRERIALgda/VaPp2r/GZf+I\nMcbEUtRqJKraICI34ZqlEoDfqeo6EbkLWK6qi71jZ4nIeqARNxqrDEBECnA1mr+3ufTTIpKHazpb\nCXwrWu/BGGNM16I6j0RVXwRebLPv9oBtBb7rPdqeu5n2nfOo6ukRD9QYY0yP2cx2Y4wxYbFEYowx\nJiyWSLrh2peubVlozxhjjGOJJIaal5FfuXIlJ5xwAhMnTmTSpEk888wz7cpGYhl5gBUrVvDSSy9F\nJH5jjAFbRj4upKWl8cQTT1BUVMT27ds59thjOfvssxkypGVuZsjLyHdlxYoVrF27llmzgi0iYIwx\n3Wc1kjgwbtw4ioqKABgxYgRDhw6lO7PxP/30U84++2yOPfZYTj75ZD755BMAFi5cSHFxMZMnT+a0\n006jurqau+66i6effrpHtRljjAnGaiTAve/dy0f7Pmq3v6ahBgB/oh+gpUwo/STHZB/D96cHXb2l\nU++99x51dXWMHTs25HPmzZvH/PnzGTt2LEuWLOGmm27ilVde4c477+Stt95i2LBhVFRUkJqayu23\n387atWv5xS9+0e3YjDEmGEskcWTHjh1cddVVPP744/h8oVUWKyoqWLp0aaslURoaGgCYOXMmV199\nNZdddhkXX3xxVGI2xhhLJNBhzaHtMvLRvDXqgQMHOPfcc/nxj3/M8ccfH/J5qkpubm7QPpNHH32U\nd999lxdeeIHJkyezevXqSIZsjDGA9ZHEhbq6Oi666KKW2kN3ZGVlMXz48JZb5TY1NbFq1SoANm7c\nyPHHH8/dd99NVlYW27ZtIzMzk4MHD0b8PRhjBi5LJHHgD3/4A2+//TaPPfZYy7De7ozKWrhwIQ89\n9BCTJ09m4sSJvPDCCwDcfPPNlJSUUFJSwplnnklxcTGnn346q1atYurUqdbZboyJCGvaiqHmpdvn\nzp3L3LlzQzon2DLyY8aMCXr/ksWL2y62DHl5eSxfvrwH0RpjTHCWSDpRU9/Y6nk0+kaMMaavs0Ri\nTIys33Eg1iEYExHWR2KMMSYslkiMMcaExRKJMcaYsFgi6YYtV13NlquujnUYxhgTVyyRxFBvLyO/\naNEi7rvvvojFb4wxYKO24kIkl5FvaGggMTH4j/Wiiy6KfPDGmAHPEkkcGDduXMt24DLygYmkMyee\neCKnnHIK77zzDhdffDGFhYX89Kc/pa6ujry8PJ566imGDh3K/PnzW1b+nTt3Ljk5OSxbtoydO3dy\n//33W6IxxvSIJRJg509/Su2H7ZeRp74agC1JqQDUfOTKhNJPkjL+GI74wQ+6HUtPlpEHt+jj22+/\nDUB5eTnnn38+IsJDDz3E/fffz7333tvunN27d7NkyRLWrFnD7NmzLZEY04c0NjVSXlvOvpp9lFWX\ntf63powa2QY0sKtyF8PSh0U1FkskcaQny8g3mzNnTsv21q1bmT17Njt37qS2trZVjSfQhRdeiIgw\nadIktm3bFlbsxpjw1TTUUFZTxr5qlwzaJofA/eU15Sja7hqJkki2PxulASGB+qb6qMdtiQQ6rDl8\nuHcDAEfmHgUcrokc+eQTEY+hp8vIN0tPT2/ZvvHGG/nBD37AOeecw2uvvcY999wT9JyUlJSWbdX2\nX0hjTHhUlQN1ByirLqOspqwlGQRLDGXVZVQ1VAW9TnpSOtn+bHL8OYzOHM3UoVPJ9me7fak5Lf/m\n+HMYlDwIEWHGAnePovzM/Ki/z6gmEhGZBfwSSADmq2q732giMhu4A1BglapeKSKnAQ8EFDsGmKOq\nz4lIIbAQyAZWAFepal0030e0hbOMfDD79+9n5MiRqCqPP/54BCI0xjSrb6xnX01AMuigaak5YTRo\nQ7trCEKWP6slARTnFpPjzzmcFPyHk0OWP4vUxNQYvNPQRS2RiEgC8CDwJaAUWCYii1V1fUCZIuA2\nYKaqlovIUABVfROY4pXJBjYAr3in3Qs8oKoLReQh4DrgN9F6H72heRn5srIyHnvsMYCWJeV74o47\n7uCiiy4iPz+f6dOns2PHjghGa0z/oqpUNVS1SgaBtYe2yeJAXfA10lISUloSwLC0YYzPHt+uxtCc\nJIakDCHBl9DL7zR6olkjmQ5sUNWNACKyELgAWB9Q5hvAg6paDqCqu4Nc51Lgb6paJSICnA5c6R17\nHFeb6ZOJJFLLyP/jH/9o9fySSy5pdevdZtdff33L9lNPPRU0FmP6g8amRipqK0KqMZTVlFHbWBv0\nOoOSB7UkgaKsIpcoUl0yaFWDSM0hLTEN9ytq4IlmIhkJfB7wvBSY0abMOAARWYJr/rpDVV9qU2YO\n8HNvOweoUG2pK5Z6r9OOiMwD5gGMHj26h2+htWj0jRhjQlPTUNMqGTQngeYaROCxitoKmrSp3TWa\nO6Kbk0Hh4MKgNYbm/oekhKQYvNO+J5qJJFhqbtujmwgUAacC+cA7IlKsqhUAIjIcKAGa79oUyjXd\nTtVHgEcApk2bZj3JxsSZlo7ojkYpBdYeavZRWV8Z9DppiWktSWB05mimDJ3Sqo8hsCM6MzkTn9iC\nHpEWzURSCowKeJ4PbA9SZqmq1gObRORjXGJZ5h2fDSzyjgPsBYaISKJXKwl2zZCp6oCtiobCRnKZ\n7qpvqqe8przDIautkkTtPhqauuiI9vf9juiBIJqJZBlQ5I2y2oZrorqyTZnngCuAx0QkF9fUtTHg\n+BW4zngAVFVF5E1cv8lC4Brg+Z4E5/f7KSsrIycnx5JJEKpKWVkZfr8/1qGYGGruiG5OBs1NScFq\nDGXVZR12RCf7kltqBXlpeRyTfUzQjuhsfzZZKVn9qiN6IIhaIlHVBhG5CdcslQD8TlXXichdwHJV\nXewdO0tE1gONwC2qWgYgIgW4Gs3f21z6+8BCEfkx8AHw257El5+fT2lpKXv27OmwzI5D3rE90Z/Q\nE4/8fj/5+dEfg256V3NHdGc1hsDtmsaaoNcJ7Ig+ashRzDhiRquO6ObtbH826Unp9gdbPxbVeSSq\n+iLwYpt9twdsK/Bd79H23M0E6Uj3RoFNDze2pKQkCgsLOy3ztQU/AuDda/8U7ssZE1W1jbVBawrB\nag2hdkQXDCpoVVNo27RkHdGmmc1sNyYONXdEd9SE1HZ/KB3R+Rn5TM6bfDgZtBnGah3RpqcskXSi\nnnJU6rjrX3cBrhMwUGdV9cCybcuFeqzdNQPKtrpGJ3G1u57Q4bFIx9XlsdbBdBxLd67Zwc+kp3GF\n+jPuSVz1lAPKz5f/POgEuFA6oifmTAw+fDXV1SKsI9r0BksknWiSGhqp4o2tbwRdHK1Z29FNgWXb\nnhdYtt01lQ6PtbpmZ9cINa5uxNyTuLqKzdByW7mnP3y6JQnkpeVxdPbRQYevWke0iVeWSDqRosMB\neOty6yOJtM6SYafHWme1Do+FnDQ7GeLc07g6vWbAsbN+/zUA3r16kXVEmz7NEomJiVCb3/ozwdUs\nLImYaGlqTKKpSfH5ovsds0RijDF9WE19I1v3VbFxTyWb9layae8hNu+tYs/nc2hqTKO0vJrROWlR\njcESiTHGxLnGJmVbeTWbyirZtOcQm/ZWsnGvSxzbKqoJbE3NzUhhTG46yRmfk5h0gLSUM6MenyUS\nY4yJA6rKnkO1bGqpWRxOFlvLqqhrPDz3JyMlkTF56Rx7ZBaXHptPYW46hbnpFOSmM8jv5vfMWPCf\ngEss0WaJxBhjetH+6no2t0kUzc1Rh2oPD/lOTvBRkJvGmNx0zhg/lDG56RTmZlCYm05uRnJc9a1Z\nIjHGmAirqW9kS1kVm/YeYtPe5n9d0th76PANXUUgPyuVwtwMph2Z3VKzKMxNZ8SQVBKi3EkeKZZI\njDGmB5r7LTYGJIlNeyvZuKeS7ftb91vkZaZQmJvOmeOHtUoWo7LT8Cf1/XlBlkiMMaYDqsqeg7UB\nTVCV3uioQ2zdV0V94+Fsken1WxxXkEVBruu3GJObQUFuGpn+/r0umSUSY8yAt7+6vqWvYtOeSjY1\nN0vtqaSyrrGlXHKij4KcNI4amsGXJhzh+i3yXO0iJz2++i16U0iJRERmAitVtVJE5gJfAH6pqlui\nGp0xxkRITX0jm8sq2dzcyR0wOqqs8nC/hU8gPyuNwtx0ph2ZzRgvURTk9K1+i94Uao3kN8BkEZkM\n/B/cPUCeAE6JVmDGGNNdDY1NbKuobpcoNu1t328x1Ou3OGviMApyXLIYk+f6LVIS+36/RW8KNZE0\neHcnvABXE/mtiFwTzcCMMSYYVWX3wdpWM7mbh9J+3rbfwp/ImFzXb1GYO4rCvHTGePMtMlKsZT9S\nQv0kD4rIbcBc4GQRSQD6d++RMSam9lfVt4yI2txqzkUlVW36LQpz0hk3NJOzJx7hdXK7Gkb2AO63\n6E2hJpLLcfdbv05Vd4rIaOC+6IVljBkImvstNu1pnSg27a1kX5t+i1HZrt9iemF2S62iMDedEYNT\no74ooelcyDUSXJNWo4iMA44Bfh+9sOLDgV0nUF+dx5WPLmWQP4lBqYlk+pPabCcyKDWJTH+itz+J\nzJRE+2Ib42lobKK0vDpgJrc372JPJdv3t74f/LBBrt/i7InN8y3cTO7R2WkkJ9rdG+NVqInkbeAk\nEckCXgeW42opX41WYPHAl1iFL7GKuoYmNu49xMGaBg5U17caDtiRzJS2CaaTxBNk2/7TmL5EVdl1\noPbw5Lzm/osyt05UQ9PhfotB/kQK8zKYMSan1eQ867fou0L9qYmqVonIdcB/q+r/E5GV0QwsHmTk\nrALg2WtvaLW/obGJQ7UNHKhu4EBNvXt4283Jpu32jv01fLzrIAeqGzhYU09TFzcP9Cf5WiWeQX4v\nybTbPpyo3H63nZqUYG3DJuIqqurY6PVZbAoYRru5rHW/RUqij8LcdI4elsms5n6LPDeE1vot+p+Q\nE4mInICrgVzn7Ruw4+MSE3wMSUtmSFpyj85XVSrrGjlQ7SWbmvpOtt2/FdX1fL6vqmVf4EqgQWP0\nSfvEY81zJgTVdV6/RZuZ3Jv2VlJeVd9SLsEnjMpKpTA3nePH5FCYm+aaovLSGT7Ib9+fASTURPId\n4DZgkaquE5ExwJvRC6t/ExEyUhLDqsbX1De2qfW4ms7hmlH7WlKkm+c6qyVlWvNcXKtv6bc4FDCM\n1tU0Ouq3mFU8vGU0VGFeOqOyrN/COCH9JlPVvwN/F5FMEclQ1Y3Av3d1nojMAn6Jq73MV9V7gpSZ\nDdyBuwP3KlW90ts/GpgPjPKOnaOqm0XkMdxEyP3eJb6mqv2+ma0tf1IC/qQEhmb27Pzeap7rONlY\n81y0qSo7D9S0jIjaHDAiauu+9v0WY/IyvJrF4WU/CnLSSbd+C9OFUJdIKcHNZM92T2UPcLWqruvk\nnATgQeBLQCmwTEQWq+r6gDJFuJrOTFUtF5GhAZd4AviJqr4qIhlAYFvOLar6bGhv0QQTrea5zmpJ\n1jwXHc39Fm1viLR5byXV9Ydrnv4kHwU56RwzPJMvlxzhjYhyzVFZaUmWtE2PhfqnxsPAd1X1TQAR\nORV4FPhiJ+dMBzZ4tRdEZCFwAbA+oMw3gAdVtRxAVXd7ZScAiar6qrf/UKhvyPSOvtA8JwIZye2b\n50KtJcVT81xVXQOb91a1zOQOnHNR0abfYnR2GgU5aZwwJqdlJndhbjpHWL+FiZJQfwukNycRAFV9\nS0TSuzhnJPB5wPNSYEabMuMARGQJrvnrDlV9ydtfISJ/BgqB14BbVbX5N8dPROR23FDkW1W1NsT3\nYeKINc+1Vt/YxOf7qtrVKjbtrWRHm36LIwb5KcxN55ySgH4L7/4WSQnxkfzMwBFqItkoIv8XeNJ7\nPhfY1MU5wf73tP1vmwgUAacC+cA7IlLs7T8JmApsBZ4BvoZbLPI2YCeQDDwCfB+4q92Li8wD5gGM\nHj26i1BNX9TXm+cO7p4OwNcfW9bSb9EYkNkGpyYxJi+dE8bmtNxmtSA3zfotTNwJ9dv4deBO4M+4\nBPE2cG0X55TiOsqb5QPbg5RZqqr1wCYR+RiXWEqBDwKaxZ4Djgd+q6o7vHNrRWQB8L1gL66qj+AS\nDdOmTevi704zEMW6ea76wNEosMNfw/jhmZzT0m/hmqOy0nuWII3pbaGO2ionhFFabSwDikSkENgG\nzMGt1xXoOeAK4DERycU1aW0EKoAsEclT1T3A6bjZ9IjIcFXdIa694EJgbTfjMiZiwmmem7HgEgD+\ndu2fIhyVMb2r00QiIn+hfXNUC1U9v5NjDSJyE/Ayrv/jd94clLuA5aq62Dt2loisBxpxo7HKvNf+\nHvC6lzDex3XuAzwtInm4mtFK4FuhvVVjjDHR0FWN5D/Dubiqvgi82Gbf7QHbCnzXe7Q991VgUpD9\np4cTkzHGmMjqNJF4ExGNMcaYDoU6IXEN7Zu49uP6LX7c3BzV3xxZ/1msQzDGmLgX6nCVv+H6MP7H\nez4H10exH3gM+ErEIzPGGNMnhJpIZqrqzIDna0RkiarOFJG50QjMGGNM3xDqFNgMEWmZlS4i04EM\n72lDxKMyxhjTZ4RaI7ke+J23eKIAB4DrvGVSfhat4IwxxsS/UCckLgNKRGQw7m6JFQGH/xCVyIwx\nxvRYQV3QRT+iIqSmLREZLCI/xy2S+JqI3O8lFWOMMQNcqH0kvwMOArO9xwFgQbSCMsYY03eE2kcy\nVlUvCXh+p4gMuLsSGmOMaS/UGkm1iJzY/EREZgLV0QnJGGNMXxJqjeQG4PHmznZgH+7+IMYYYwa4\nUEdtrQQmi8gg7/mBqEYVJwrrajnks7vNGWNMZ7paRr7dqrzefgBU9edRiClu3F5WQZpWwcKvwvE3\nwJEz3Y3AjTHGtOiqRtLDu2n3D6WJo8luLCNvyz/hoxdgWAkc/y0ovhSS/LEOzxhj4kJXy8jf2VuB\nxKMGSWJ34hHkffdVWP0HePcheP5GePU/YNrX4bjrIPOIWIdpjDEx1e0OABFZEY1A4lpSKhx7Ddzw\nT7j6ecg/Dt6+Dx4ohj/Pg+0fxDpCY4yJmVBHbQUauJ0EIjDmVPco+wzeewQ+eApWPwOjjnf9KMec\nBwk9+VjNQNObS1gYE009GZL014hH0RfljIUv3wvfXQ9n/wwO7oA/XgP/NQWW/BKqy2MdoTHG9Ipu\nJxJV/VE0Aumz/IPhhH+Df/8A5vwPZBXAq7fDzyfACzfDno9jHaExxkRVqLfaPUjHt9r936q6MdKB\n9Tm+BDjmXPfYucZ1zH/wNCz/HYw9wzV7jT0DbF6KMaafCbUx/+fAdtytdgV3q90jgI9xCzqeGo3g\n+qwjSuCCB+HMO2H5Alj2KDx9KeQUwYxvwuQrICWj6+sYY0wfEOqfx7NU9WFVPaiqB1T1EeAcVX0G\nyIpifH1bei6ccgt8Zy1c/KhLHi9+Dx6YAK/8CCq2xjpCY4wJW6g1kiYRmQ086z2/NOBY2yYv01Zi\nMkyaDSWXwefvwdJfw79+Df960I3yOv7fYPTxNmveGBMxz3zzhF57rVBrJF8FrgJ2A7u87bkikgrc\n1NFJIjJLRD4WkQ0icmsHZWaLyHoRWSci/xOwf7SIvCIiH3rHC7z9hSLyroh8KiLPiEhyiO8h9kRg\n9AyY/Th8exV88d9h09uwYBY8cgqs/D001MY6SmOM6ZaQEomqblTVr6hqrqrmedsbVLVaVf8R7BwR\nSQAeBL4MTACuEJEJbcoUAbcBM1V1IvCdgMNPAPep6nhgOi6JAdwLPKCqRUA5cF3I7zaeDBkFX7rT\nDR8+7wGor4HnvuUmOb51Dxza3fU1jDEmDoR6q91xIvK6iKz1nk8Ska6GAU8HNnhJqA5YCFzQpsw3\ngAdVtRxAVXd7158AJKrqq97+Q6paJW61yNM53MT2OHBhKO+hJ+7KuY+7cu6L1uWd5HS33MqN78Lc\nP8PwyfDWz+CBibDoBtixKrqvb4wxYQq1aetRXM2hHkBVV+NGbnVmJPB5wPNSb1+gccA4EVkiIktF\nZFbA/goR+bOIfCAi93k1nBygQlUbOrkmACIyT0SWi8jyPXv2hPg2W3vmmyf0XjujCBx1Bsx9Fm5a\nDl+4GtY/Bw+fDAvOgQ//Ak2NvROLMcZ0Q6iJJE1V32uzryFoycOC9Ry37ZhPBIpww4evAOaLyBBv\n/0nA94DjgDG4G2mFck23U/URVZ2mqtPy8vK6CDXO5BbBufe7Zq8v3Q0Vn8Mzc92s+X/+CqorYh2h\nMca0CDWR7BWRsXi/tEXkUmBHF+eUAqMCnufj5qK0LfO8qtar6ibcvJQib/8HXrNYA/Ac8AVgLzBE\nRBI7uWb/kZoFM//dzZqf/SQMyodXfuhmzb94C+zdEOsIjTEm5ERyI/AwcIyIbMN1in+ri3OWAUXe\nKKtkXFPY4jZlngNOAxCRXFyT1kbv3CwRaa5KnA6sV1UF3uTw8ONrgOdDfA99V0IiTDgfvv43mPd3\nt/3+Y/CrY+Hpy+CzN0BtFLYxJjZCTSTbgAXAT3Cd5q/ifol3yKtJ3AS8DHwI/EFV14nIXSJyvlfs\nZaBMRNbjEsQtqlqmqo24Zq3XRWQNrknrUe+c7wPfFZENuD6T34b4HvqHEVPgoofcJMdTbnVL2D95\nEfz6eDeLvq4q1hEaYwYY0RD+khWRl4AKYAXQ0uOrqvdHL7TImTZtmi5fvjzWYURHQy2s/RMs/Q3s\nXO2aw45D+y0ZAAAb40lEQVT9Ghz3DRgcdByCiROXP/wvoHcnjhnTHSLyvqpO66pcqDPb81V1VtfF\nTK9LTIEpV7r1u7b8E979jVvGfsl/wYQL3GKR+cfZrHljTNSEmkj+KSIlqromqtGYnhOBgpnuUb7F\n3XRrxZOw7s8w8liYcYNLLIl9ZyEAY0zfEGofyYnA+95yJ6tFZI2IrI5mYCYMWUfC2T9xw4fP+U+o\n2Q9/vh5+UeJuEVy5N9YRGmP6kVBrJF+OahQmOlIyYPo3YNp1sOE11+z1xo/h7/e5RSSPvwGGTYx1\nlMaYPi6kRKKqW6IdiIkinw/GneUeuz9yN91atRA+eBIKT3bNXuPOdjfnMsaYbrLb9Q00Q4+Br/zC\nNXudeQeUfQYLr4D/PtaN/Ko5EOsIjTF9jCWSgSotG0682S1nf+kCyBgKL93qZs3/7VbYZ3dPNsaE\nxhLJQJeQBMUXw3WvwDfegKO/7G4N/F9fgN9fARv/brPmjTGdskRiDht5LFzyqJs1f/L34PN34Ynz\n4TczYcUTUF8d6wiNMXHIEolpb9BwOP1HcPM6OP9Xbt/i/+XukfL63XCgq/U6jTEDiSUS07GkVPjC\nVXDDErjmLzBqBrxzP/yiGP50PZS+H+sIjTFxINR5JAPSlquuBuDIJ5+IcSQxJuKGCRee7Drh33vU\nzZpf80fInw7HfwvGn+/6W4wxA44lEtM92WNg1s/g1Ntg5f+4OSnPfh0GjYTjrncLRqZlxzrKPsEW\nazT9hTVtmZ7xD3I1kf/1PlyxEHLGwut3uuHDf/m2m/hojBkQrEZiwuNLcEOGj/4y7FrnJjWu/L27\n8daY0+D4f4OjznSz640x/ZL97zaRM2wiXPArN2v+9B/Bno/gfy6DB49z/Sq1h2IdoTEmCiyRmMhL\nz4WTb4Fvr4ZLfgv+wfDi91yz18s/hPLNsY7QGBNBlkhM9CQmQ8mlbsb8da/BUWe4pq//mgoLvwqb\nl9iseWP6AesjMb1j1HEwagHsL4Vl810fykcvwBElrh+l+BJ3t0djTJ9jNRLTuwbnu1WHb14P5/0C\nGuvhuRvcrPk3fwYHd8U6QmNMN1kiMbGRnAbTroV/WwpXLYIRX4C/3+MSyqJvwfaVsY7QGBMia9rq\nRP327TTV1lK+8Bn8JcX4x41Dkmz2dkSJwNjT3WPvBnjvYfjgaVj1exh9gruL49HnQoJ9VY2JV6ID\noLNz2rRpunz58m6f9+nJp9BQVgaNjQBIcjIp448htbgEf0kxqSUlJBcWIjZHIrKqK+CDp1xSqdgK\ng0fB9Hlu3a/UrFhHZ8yAISLvq+q0LstFM5GIyCzgl0ACMF9V7wlSZjZwB6DAKlW90tvfCKzxim1V\n1fO9/Y8BpwD7vWNfU9VO20F6mki2XHU1qsqIn/6E6jVrqFmzluq1a6hZtx6tdkuq+zIy8E+cSGpJ\nMf7iElJLikkcMQIR6fbrmTaaGuHjF2HpQ7DlH5CUBlOuhBnfgtyiWEdnTL8XaiKJWnuBiCQADwJf\nAkqBZSKyWFXXB5QpAm4DZqpquYgMDbhEtapO6eDyt6jqs9GKPZCIkDx6NMmjRzP43HMB0IYGaj/b\nSM3aNS0JpuzxJ6C+HoCE7GxXYwmouSTm5PRGuP2LLwHGf8U9dqx263qteMKN+jrqS26JlrFnuOYx\nY0zMRLPheTqwQVU3AojIQuACYH1AmW8AD6pqOYCq7o5iPBEjiYn4jx6H/+hxDLnkEgCa6uqo/eij\nVjWXyrffaZknkThieKvE4p84kYTMzFi+jb5l+CS48NduxNfyBS6ZPHUJ5I5zNZTJcyA5PdZRGjMg\nRTORjAQ+D3heCsxoU2YcgIgswTV/3aGqL3nH/CKyHGgA7lHV5wLO+4mI3A68DtyqqrXReAPd4UtO\nJnXSJFInTWrZ13iokpr16w43ia1Zy8FXXmk5nlxY2Krm4h8/Hp/fH4vw+46MoXDq9+HE78C6RbD0\n1/DX77oFI4/9Ghz3DRgyKtZRGjOgRDORBGtvaNshkwgUAacC+cA7IlKsqhXAaFXdLiJjgDdEZI2q\nfoZrCtsJJAOPAN8H7mr34iLzgHkAo0ePjsw76qaEjHTSp08nffr0ln0N5eXUrF3bUnOp/Ne/OLD4\nL+5gYiIpRUWkFhe31FxSioqQRBux1E5iiquFTLocti6Fd38D//xv+OevYPx5bpLjqBnW7GVML4jm\nb6hSIPBPw3xge5AyS1W1HtgkIh/jEssyVd0OoKobReQtYCrwmao23+e1VkQWAN8L9uKq+ggu0TBt\n2rS4GZqWmJVFxkknkXHSSQCoKg27drUklpq1azjw8stU/PGPAEhKCv7x4/GXlJA6qQR/cTHJRx5p\nI8WaicCRJ7hHxVbvpluPw/rnYfgUl1AmXuSWazHGREXURm2JSCLwCXAGsA1YBlypqusCyswCrlDV\na0QkF/gAmAI0AVWqWuvt/xdwgaquF5HhqrpD3LCoB4AaVb21s1jCGbUFvX+HRG1qon7rVqq9xFK9\nZi0169ejNTUA+DIz8RdPbN2Zf8QRNlKsWV2lm4ey9CEo+xQyhnk33boWMvJiHZ0xfUbMR22paoOI\n3AS8jOv/+J2qrhORu4DlqrrYO3aWiKwHGnGjscpE5IvAwyLShJt9f0/AaK+nRSQP13S2EvhWtN5D\nrIjPR3JBAckFBQz+ynlA80ixz6hZ4yWWNWsoW7AAGhoASMjNbdUk5i8pITFrgM65SE73EsfX4bM3\nXD/Kmz+Bt/8TSi5zo72OKIl1lMb0GzYhsRPxfs/2ptpaN1Js9ZqWmkvdpk0tI8WSRo50TWLeHBf/\nxIkkZAzQkU17PoZ3H3Y1lfoqKDjJjfY6+stumLExpp24mJAYL/prIgmm8dAhatauO9wktmYN9du9\nrikRkseM8WouLsGkHHMMvpQBtOpudbmbi/LuI3CgFIYcCTO+CVPnuvumGGNaWCIJ0NNE0l807NvX\nqkmseu1aGsvK3MGkJPxFRYdrLiUlpIwd2/9HijU2uGXsl/4GPl8KyRkw5asuqeSMjXV0xsQFSyQB\nBnoiaUtVadixo3Vn/tq1NB1yt8KV1FT8Eya0qrkkjR7dfzvzt61ws+bX/hmaGmDc2a7Za8ypNnzY\nDGiWSAJYIumaNjVRt3lLqyaxmg8/RGvdXE/f4MGkTpzYquaSNGxYjKOOsIM7YfnvYNlvoWov5I13\nHfOTLoek1Mi/3gK35A7X/jXy1zYmAiyRBLBE0jNaX0/thg0By76spfaTT1pWQ07My2vdmV88sX+M\nFKuvgbV/cpMcd66B1Gxv1vz1MHhk5F7HEomJc5ZIAlgiiZymmhpqPvyw1bIvdZs2tRxPGjWq1UrI\n/gkT8KX30ZFiqrBlietH+eivbnTXhAtgxg3u1sHhskRi4lzM55GY/snn95M2dSppU6e27Gs8eJCa\ndetczWX1Gqo+WMmBF//mneAjZewYV2NpXvbl6KPxJfeBmeYiUHCie5Rv9mbNP+FqKyOnuZtuTbgA\nEuxmZ2ZgsxqJiYqGvXtb38NlzVoay8sBkKQkUo4+2lvyxdVckseMQRL6wHyO2oOw8veu2WvfRsgc\nAcdd52bNp3fzVgFWIzFxzpq2AlgiiT1VpX7b9lb3cKlZt46mykoAJC2N1AkTWnfm5+fH70ixpibY\n8KqbNb/xLUj0w6TZrtlr2ITQrmGJxMQ5a9oycUVESM4fSXL+SAbNmgV4I8U2bWpVcyl/+mn21dUB\nkDBkCP7AZV+Ki0kaOrSzl+k9Pp8bJjzubNj9oRs+vGqha/oqPMU1exWd7coZ089ZjcTEFa2ro+bT\nT1s1idVu2HB4pNiwYa3vPllcTMLgOJmRXrUP3n/M9aUc3A7ZY2D6N2HqVyGl/U3MtnzJ9TMd+eoH\nvRyoMaGxpq0Alkj6tqbqam+k2OE5LnVbtrQcTzpydOu7T44fjy8tLXYBN9bDh4vdaK/SZZAyyC3B\nMn0eZBe2FLNEYuKdJZIAlkj6n8b9+72RYodn5zfs3OkO+nykHHVUQJNYCf5xRUgsRoqVLncJZf1z\n0NQIR5/jmr0KTmTLWV8ALJGY+GWJJIAlkoGhfvfu1gtWrl5N4/79AEhyMinjj2lVc0kuLOy9G4Qd\n2O7uM798AVTvg2HF7P3HLir3pHPkK6t6JwZjuskSSQBLJAOTqlJfWtp6wcr169GqKgB86en4J05s\nVXNJGjkiuiPF6qthzR9dLWX3erQJZEi+u+FWxjBI9/7NGOoe6UO953muiSxeR7GZfskSSQBLJKaZ\nNjZSt3Fjqyax2o8+QuvrAUjIymrdmV9SQmJubhQCUXZdNh7/kCoGn3suHNoFlbvh0G6o3APa1P6c\nRL+XWIa2STTNj4BElJIR+ZjNgGOJJIAlEtMZrauj5uNPWi1YWfvZZ26uCJA4fHirlZD9xcUkZLYf\nhdVdHXa2NzW6EWCVu12CObSndaJpflTuhsq9QJD/w0lpwRNN21pO+lBIjuHABBM1kbifks0jMSZE\nkpxMakkxqSXFZF3h9jVVVlLz4Yet7uFy8NVXW85JLihovWDlhPH4/P7IBORL8Jq68mDYxM7LNjZA\nVVnHiebQLij7DLb80/XNBJOcGaSWMyyguS3gWOIAuglaX7dzTa+9lCUSY4LwpaeTNm0aadMO/zHW\nWFFBdUBnftW773LgL39xBxMSSCkqOrxg5aQSUo46CkmK8jpcCYmQOcw9utJY75rN2iaalhrPHje5\n8tDfoaYi+DX8g4MkmiD9Oul5kNgH1lMzEWGJxJgQJQwZQsaJM8k4cWbLvvpdu1st+3LglVep+OOz\nAEhKCv7x41vVXJILjuy9kWLt3kASDBrhHl1pqPWSTidNazvXwKHXofZA8GukZnU9gCBjGKTluoRo\n+iz76RkThqRhQ0kadgaZZ5wBeCPFPv+81bIvFc8+S/mTTwLgy8jAX+ya0RqqGvElCfW7duNLS8WX\nmho/tzhOTIHB+e7Rlfrqw4MEDu0KXuPZ9r47XncoyAUE0nK6HkCQMdSV8/WBxT0HGOtsNybKtKGB\n2s82tl6w8pNPwBspFkiSk11CSU/Dl5qGLzUVX5r7V9Kat9Pcv83Jp2Xf4bK+tDQkYJ+kpMTHAph1\nlZ03rQUmoobq9ueLz9VgumpayxjmakQDeK2zSKycYJ3txsQJSUzEf/Q4/EePY8gllwDQVFfH5jOP\nQxuU7H//IU3V1TRVV6FVVTRVVdNUVdWyr6mqioY9e9zz5v1VVdDQEHoQPh8+v/9wgmpOOKmp+NLT\nkNTU1vvb7vOSlktQqfjS0ntWi0pOd8vEBCwVE5Sqq720NKUFSTSVu2Hvp267sTbIe050SSZoomlT\n40nNsjk6YbBEYkwM+JKTSUjxQQpkzbm8R9fQurrWyaWyyiWj5n1V1QHHA5JUwL6mysp2SUqrg9QE\nOtFSi0pLa1UjaleL8hJUR7WoVgkqLQ1JyUByMiFnbBcfhELN/s6b1g7tgt3r3b9NQRJwQrKXcEIY\nMm0TQ9uJaiIRkVnAL4EEYL6q3hOkzGzgDtxg+FWqeqW3vxFoHr+2VVXP9/YXAguBbGAFcJWq1kXz\nfRgTjyQ5mYTk5IivfqxNTS4ZVQckHe/Rsr+yKvq1KJEOm/mC1qLSAhPUGHxpE/GNDKxFeeWoQWr2\ntRk8EFDjObgDdqzyJoY2to/LJoa2E7VEIiIJwIPAl4BSYJmILFbV9QFlioDbgJmqWi4igTebqFbV\nKUEufS/wgKouFJGHgOuA30TrfRgz0IjPh6Sn40tPj/i147oWlZqKLy0PSRvt9vlT8SWBL7EJn9Qj\nUouPGnxaia/hIL69+5GdG/HVL8NXtw9fYhOSoK0rKx1NDA3W3NaHJ4ZGs0YyHdigqhsBRGQhcAGw\nPqDMN4AHVbUcQFV3d3ZBcb2FpwNXersex9VmLJEY0wf0z1pUEuDN4xHB509GUpLwJfnwJYlLRL59\niG8nvuZklKgtD0lsctt+P77MQfgys/ENzkGG5OEbMgxfzgh82SOQwSMOr0aQFKHJrxESzUQyEvg8\n4HkpMKNNmXEAIrIE1/x1h6q+5B3zi8hyoAG4R1WfA3KAClVtCLjmyGAvLiLzgHkAo0ePDv/dGBNp\nR5TEOoJ+I+5rUYFJq6qSpqpqtKYmyKvVA7u8R9v3GJB4knz4UhLxpSTjS/V7Nat0l4gyhuAbnE1j\nVR2qPhr374/6zd+imUiC9Ua1HWucCBQBpwL5wDsiUqyqFcBoVd0uImOAN0RkDRBs5lPQ8cuq+gjw\nCLjhvz17C8aYgS6mtaiDB2jav5em/XvRA/toOlhB06EDXjKqoqm6hoby/TTt3EdTvdLUIDQ1CGjz\nr98mGj5dRsK0MyMae1vRTCSlwKiA5/nA9iBllqpqPbBJRD7GJZZlqrodQFU3ishbwFTgT8AQEUn0\naiXBrmmMMXEv4rWo+pqWAQRavp29P7kZn6+J5HHBupojK5qzdZYBRSJSKCLJwBxgcZsyzwGnAYhI\nLq6pa6OIZIlISsD+mcB6dbMn3wQu9c6/Bng+iu/BGGP6hiQ/DBkN+dOQkvOpKhvEoT1DkEFRuA1C\nG1FLJF6N4SbgZeBD4A+quk5E7hKR871iLwNlIrIelyBuUdUyYDywXERWefvvCRjt9X3guyKyAddn\n8ttovQdjjDFdi+o8ElV9EXixzb7bA7YV+K73CCzzTyBoT6Q3Cmx6xIM1xhjTIzaz3ZgYCeeGQ8bE\nk4G7opkxxpiIsERijDEmLJZIjDHGhMUSiTHGmLBYIjHGGBMWSyTGGGPCYsN/jTGmHzryyhG99lqW\nSIwxpj+69q+99lLWtGWMMSYslkiMMcaExRKJMcaYsFgiMcYYExZLJMYYY8JiicQYY0xYLJEYY4wJ\niyUSY4wxYbFEYowxJizi7nbbv4nIHmBLD0/PBfZGMJxIsbi6x+LqHoure/prXEeqal5XhQZEIgmH\niCxX1WmxjqMti6t7LK7usbi6Z6DHZU1bxhhjwmKJxBhjTFgskXTtkVgH0AGLq3ssru6xuLpnQMdl\nfSTGGGPCYjUSY4wxYbFEAojIfSLykYisFpFFIjKkg3KzRORjEdkgIrcG7C8UkXdF5FMReUZEkiMU\n12Uisk5EmkQk6MgLETlaRFYGPA6IyHe8Y3eIyLaAY+f0Vlxeuc0issZ77eUB+7NF5FXv83pVRLJ6\nKy4RGSUib4rIh17Zbwcci/Xn1dvfry5/DiJyWpvvV42IXOgde0xENgUcm9JbcXnlGgNee3HA/lh+\nXlNE5F/ez3u1iFwecCyin1dH35eA4yne+9/gfR4FAcdu8/Z/LCJnhxMHAKo64B/AWUCit30vcG+Q\nMgnAZ8AYIBlYBUzwjv0BmONtPwTcEKG4xgNHA28B00IonwDsxI39BrgD+F4UPq+Q4gI2A7lB9v8/\n4FZv+9Zgn3e04gKGA1/wtjOBTwJ+jjH7vGL0/erWzwHIBvYBad7zx4BLo/B5hRQXcKiD/TH7vIBx\nQJG3PQLYAQyJ9OfV2fcloMy/AQ9523OAZ7ztCV75FKDQu05COPFYjQRQ1VdUtcF7uhTID1JsOrBB\nVTeqah2wELhARAQ4HXjWK/c4cGGE4vpQVT/uxilnAJ+pak8nX4akB3G1dQHuc4Je/rxUdYeqrvC2\nDwIfAiMj8frhxEUMvl90/+dwKfA3Va2K0Ot3pMffj1h/Xqr6iap+6m1vB3YDXU7o64Gg35dO4n0W\nOMP7fC4AFqpqrapuAjZ41+sxSyTtfR34W5D9I4HPA56XevtygIqARNS8PxbmAL9vs+8mr4r9u0g1\nIXWDAq+IyPsiMi9g/zBV3QHuFzswtJfjAsCr6k8F3g3YHavPKxbfr+7+HIJ9v37ifV4PiEhKL8fl\nF5HlIrK0ubmNOPq8RGQ6rrbwWcDuSH1eHX1fgpbxPo/9uM8nlHO7JTGck/sSEXkNOCLIoR+q6vNe\nmR8CDcDTwS4RZJ92sj9icYV4nWTgfOC2gN2/Ae724rkbuB+XKHsrrpmqul1EhgKvishHqvp2iOdG\nMy5EJAP4E/AdVT3g7Y7l59Xr369Qr+FdZzhQArwcsPs2XFNqMm6Y6feBu3oxrtHe92sM8IaIrAEO\nBCkXq8/rSeAaVW3ydvf48wr2EkH2tX2fUflOBTNgEomqntnZcRG5BjgPOEO9hsQ2SoFRAc/zge24\ndWyGiEiil/Wb90ckrm74MrBCVXcFXLtlW0QeBV7ozbi8qj2qultEFuGqz28Du0RkuKru8P7D7e7N\nuEQkCZdEnlbVPwdcO5afV69/v0SkOz+H2cAiVa0PuPYOb7NWRBYA3+vNuAK+XxtF5C1c7fJPxPjz\nEpFBwF+BH6nq0oBr9/jzCqKj70uwMqUikggMxvVxhXJut1jTFm70A+6vg/M7af9dBhR5I0KScdX8\nxV7SeRPXfgxwDRDyX8YRdAVtmh28L3uzi4C1vRWMiKSLSGbzNm5AQ/PrL8Z9TtDLn5fXRvxb4ENV\n/XmbYzH7vIjN96s7P4cOv1/eZ3ohkfu8uoxLRLKam4ZEJBeYCayP9efl/ewWAU+o6h/bHIvk5xX0\n+9JJvJcCb3ifz2JgjjeqqxAoAt4LIxYbteVVPjbg2gxXeo/mkQ4jgBcDyp2DG+XzGa7Jonn/GO8H\nsQH4I5ASobguwv31UAvsAl7uIK40oAwY3Ob8J4E1wGrvyzO8t+LyPpNV3mNdm88rB3gd+NT7N7sX\n4zoRV41fHfDzPifWn1eMvl9Bfw7ANGB+QLkCYBvga3P+G97ntRZ4CsjorbiAL3qvvcr797p4+LyA\nuUB9wHdrJTAlGp9XsO8LrqnsfG/b773/Dd7nMSbg3B96530MfDncz8ZmthtjjAmLNW0ZY4wJiyUS\nY4wxYbFEYowxJiyWSIwxxoTFEokxxpiwWCIxJgJE5FCY5z/rzdBGRDJE5GER+cxbRfZtEZkhIsne\n9oCZSGz6BkskxsSYiEzErb660ds1HzcDuUhVJwJfw62iXIebv3B50AsZEyOWSIyJIHHuE5G14u7F\ncrm33yciv/ZqGC+IyIsi0jz7+qt4s6RFZCwwA7e8RhO4JUBU9a9e2ee88sbEDasiGxNZFwNTgMlA\nLrBMRN7GLeFRgFv4cChu+frfeefM5PDyIxOBlara2MH11wLHRSVyY3rIaiTGRNaJwO9VtVHdIpB/\nx/3iPxH4o6o2qepO3HpQzYYDe0K5uJdg6prXMTMmHlgiMSaygi3R3dl+gGrcukjg1iWbLCKd/d9M\nAWp6EJsxUWGJxJjIehu4XEQSRCQPOBm3YN4/gEu8vpJhwKkB53wIHAWgqp8By4E7vVViEZEiEbnA\n284B9mjAcu7GxJolEmMiaxFu9eBVuNVe/4/XlPUn3ArAa4GHcXdl3O+d81daJ5brcTdX2uDdrOlR\nDt8v4jTgxei+BWO6x1b/NaaXiEiGqh7yahXv4e4guVNEUnF9JjM76WRvvsafgdu063vAG9NrbNSW\nMb3nBREZgrvV6t1eTQVVrRaR/8DdN3trRyd7NzB6zpKIiTdWIzHGGBMW6yMxxhgTFkskxhhjwmKJ\nxBhjTFgskRhjjAmLJRJjjDFhsURijDEmLP8fOwc7HR0GIl0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2cc308d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#plot results\n",
    "Cs = [0.01, 0.1, 1]\n",
    "n_Cs = len(Cs)\n",
    "number_penaltys = len(penaltys)\n",
    "test_scores = np.array(test_means).reshape(n_Cs, number_penaltys)\n",
    "train_scores = np.array(train_means).reshape(n_Cs, number_penaltys)\n",
    "\n",
    "test_stds = np.array(test_stds).reshape(n_Cs, number_penaltys)\n",
    "train_stds = np.array(train_stds).reshape(n_Cs, number_penaltys)\n",
    "\n",
    "x_axis = np.log10(Cs)\n",
    "\n",
    "for i, value in enumerate(penaltys):\n",
    "    #pyplot.plot(log(Cs), test_scores[i], label = 'penalty:'  + str(value))\n",
    "    plt.errorbar(x_axis, -test_scores[:, i], yerr=test_stds[:, i], label = value + ' Test')\n",
    "    plt.errorbar(x_axis, -train_scores[:, i], yerr=train_stds[:, i], label = value + ' Train')\n",
    "    \n",
    "plt.legend()\n",
    "plt.xlabel(\"log(C)\")\n",
    "plt.ylabel(\"neg-logloss\")\n",
    "plt.savefig(\"LogisticGridSearchCV_C.png\")\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用LogisticRegressionCV实现正则化的LogisticRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### L1正则"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegressionCV\n",
    "Cs = [1, 10, 100, 1000]\n",
    "lrcv_L1 = LogisticRegressionCV(Cs=Cs, cv=5, scoring=\"neg_log_loss\", penalty='l1', solver=\"liblinear\", multi_class=\"ovr\")\n",
    "lrcv_L1.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 23:52 start - memory overflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lrcv_L1.scores_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "n_Cs = len(Cs)\n",
    "n_classes = 9\n",
    "scores = np.zeros((n_classses, n_Cs))\n",
    "\n",
    "for j in range(n_classes):\n",
    "    scores[j][:] = np.mean(lrcv_L1.scores_[j], axis=0)\n",
    "mse_mean = -np.mean(scores, axis=0)\n",
    "pyplot.plot(np.log10(Cs), mse_mean.reshape(n_Cs, 1))\n",
    "pyplot.xlabel(\"log(C)\")\n",
    "pyplot.ylabel(\"neg_log_loss\")\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### L2正则"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegressionCV\n",
    "Cs = [1, 10, 100, 1000]\n",
    "lrcv_L2 = LogisticRegressionCV(Cs=Cs, cv=5, scoring=\"neg_log_loss\", penalty='l2', solver=\"liblinear\", multi_class=\"ovr\")\n",
    "lrcv_L2.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lrcv_L2.scores_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "n_Cs = len(Cs)\n",
    "n_classes = 9\n",
    "scores = np.zeros((n_classses, n_Cs))\n",
    "\n",
    "for j in range(n_classes):\n",
    "    scores[j][:] = np.mean(lrcv_L2.scores_[j], axis=0)\n",
    "mse_mean = -np.mean(scores, axis=0)\n",
    "plt.plot(np.log10(Cs), mse_mean.reshape(n_Cs, 1))\n",
    "plt.xlabel(\"log(C)\")\n",
    "plt.ylabel(\"neg_log_loss\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SVM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 模型训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\envs\\python3\\lib\\site-packages\\sklearn\\model_selection\\_split.py:2010: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train_part, X_val, y_train_part, y_val = train_test_split(X_train, y_train, train_size=0.8, random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### default SVC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.svm import LinearSVC\n",
    "\n",
    "SVC1 = LinearSVC().fit(X_train_part, y_train_part)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification report for classifier LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,\n",
      "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
      "     multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
      "     verbose=0):\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.32      0.01      0.03       753\n",
      "          1       0.41      0.17      0.24      2221\n",
      "          2       0.74      0.95      0.83      6897\n",
      "\n",
      "avg / total       0.63      0.70      0.64      9871\n",
      "\n",
      "\n",
      "Confusion matrix:\n",
      "[[  10  214  529]\n",
      " [  14  384 1823]\n",
      " [   7  330 6560]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "#在校验集上测试，估计模型性能\n",
    "y_predict = SVC1.predict(X_val)\n",
    "print(\"Classification report for classifier %s:\\n%s\\n\" % (SVC1, metrics.classification_report(y_val, y_predict)))\n",
    "print(\"Confusion matrix:\\n%s\" % metrics.confusion_matrix(y_val, y_predict))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线性SVM正则超参数调优"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "线性SVM LinearSVC中，需要调整正则超参数包括C（正则系数， 一般在log域（取log后的值）均匀设置候选参数）和正则函数penalty（L2/L1）\n",
    "\n",
    "采用交叉验证， 网格搜索步骤与Logistic回归正则参数处理类似\n",
    "这里用校验集（X_val, y_val）来估计模型性能"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def fit_grid_point_Linear(C, X_train, y_train, X_val, y_val):\n",
    "    SVC2 = LinearSVC(C = C)\n",
    "    SVC2 = SVC2.fit(X_train, y_train)\n",
    "    \n",
    "    #在校验集上返回accuracy\n",
    "    accuracy = SVC2.score(X_val, y_val)\n",
    "    \n",
    "    print(\"accuracy: {}\".format(accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 0.7093506230371797\n",
      "accuracy: 0.7101610779049742\n",
      "accuracy: 0.7100597710464999\n",
      "accuracy: 0.7056022692736298\n",
      "accuracy: 0.6845304427109715\n",
      "accuracy: 0.6198966670043562\n",
      "accuracy: 0.6298247391348394\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\envs\\python3\\lib\\site-packages\\matplotlib\\axes\\_axes.py:545: UserWarning: No labelled objects found. Use label='...' kwarg on individual plots.\n",
      "  warnings.warn(\"No labelled objects found. \"\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XucVXW9//HXm8EBA+9cVIRAAw1NxUYs8a4laoKX5OLs\nU1lHT6fscvpVaqeOZY8ev07+Tp1zinpkap1MEFQUFRVJTc1LMog3QI5IKXgDUVNKA+Tz++O7J4YR\nZm+GWbP25f18PNZjZq/93Xt/lsB+u75rfb9fRQRmZmYd6ZF3AWZmVvkcFmZmVpLDwszMSnJYmJlZ\nSQ4LMzMryWFhZmYlOSzMzKwkh4WZmZXksDAzs5J65l1AV+nXr18MHTo07zLMzKrK/PnzX4mI/qXa\n1UxYDB06lJaWlrzLMDOrKpKeLaedu6HMzKwkh4WZmZXksDAzs5IcFmZmVpLDwszMSnJYmJlZSQ4L\nMzMrqWbGWdSSCFi3Dt56a+u3v/0tvYcEPXqkn1va8n6+VJtdd4UDD4Q+ffL98zAzh0VZNmzo3Bd3\nZ7e3306faSk0RoyAQw6BUaM2brvtlndlZvWl7sPi1Vdh0qSOv7zXru38+/fqBdtvv/ltp51g9903\n/1zv3lt+XUfbdtulL1hIZygbNqSfW9oq+fmXX4YFC+CRR+D++2HatI3/XYcM2TQ8DjkEBg3aeOxm\n1rXqPiwaGmDNmo1f3uV8IZf7Rd67d+pmyYuUjq+anXrqxt9Xr4ZHH03hsWBB2m66KQULQL9+m4bH\nqFHwvvfl+2dgVisUrf/SqlxTU1N4bqj6s2YNPP74xvBYsACeeCJd8wHo2xcOPnjTs5CRI6GxMd+6\nzSqFpPkR0VSyncPCas3atbBo0cbweOQReOyxFCyQguKAAzY9C/GFdKtXDguzNjZsgKVLN4ZHa5C8\n8kp6XoJ99920C2vUqHRHllktc1iYlRABK1Zs2oW1YAE899zGNkOGbBoehxwCe+7pC+lWO8oNi7q/\nwG31S4LBg9M2btzG/atXb9qFtWABzJq18UJ6//7vvpC+zz6+kG61zWcWZmVovZDetgvrySc3Xkjf\nYQc46KBNz0JGjky3MptVMp9ZmHWhvn3h8MPT1mrtWli4cNMurCuugL/8JT3f2Agf+EAKjnPPhdGj\n86ndrCv4zMKsC73zzsYL6a3dWC0taVT+9OmbdneZVQKfWZjloKEh3VW1775pZgCAVavglFPgjDPg\nssvg05/Ot0azzvAlObOM9e8Pd90Fxx8Pn/kMfP/7Gy+Wm1ULh4VZN+jbF26+GSZPhosugq98xZNF\nWnVxN5RZN2lshN/8BgYMgP/8T1i5En75S089YtXBYWHWjXr0gB/9KM02fNFFaUzHddelMw+zSuZu\nKLNuJsGFF8Lll8PcuelaRuu0I2aVymFhlpPPfAZmzkyD/Y44Ap59Nu+KzLbMYWGWo/Hj4Y474KWX\nYMyYNCrcrBJlGhaSxkpaImmppAs38/yPJD1a3P5X0uttnvukpKeL2yezrNMsT0ceCffdl+6OOvLI\ntCqgWaXJLCwkNQBTgJOAkcBkSSPbtomIf4mIgyPiYODHwMzia3cFLgYOA0YDF0vaJatazfL2gQ/A\nAw+kO6VOOAFuuSXvisw2leWZxWhgaUQsi4i1wDXA+A7aTwZaV1k+EZgbEa9GxGvAXGBshrWa5W7o\nUPj979PCTKedBr/6Vd4VmW2UZVgMApa3ebyiuO9dJL0XGAbctTWvlXSepBZJLatWreqSos3y1Dra\n+9hj4Zxz4Ac/8GhvqwxZhsXmlofZ0l/7ScB1EfHO1rw2Ii6LiKaIaOrfv38nyzSrLDvsALNnp7ml\nLrgAvvpVj/a2/GU5KG8FMLjN472AF7bQdhLw+XavPabda3/XhbWZVbTGRrj66nSm8cMfptHeV17p\n9TEsP1meWcwDhksaJqmRFAg3tW8kaV9gF+DBNrvnAB+VtEvxwvZHi/vM6kaPHvBf/wXf+16aJmTc\nuI1rZZh1t8zCIiLWA+eTvuQXAzMiYqGkSyS1ndV/MnBNtFlYIyJeBb5LCpx5wCXFfWZ1RYJvfAN+\n8Ys0HuP449MUIWbdzYsfmVWJG29M1zGGDYM5c2DIkLwrslpQ7uJHHsFtViVOOy2dXbz4YhrtvXBh\n3hVZPXFYmFWRo46Ce++F9evTaO8HHsi7IqsXDguzKnPggSkkdtstjfaePTvviqweOCzMqtCwYWkO\nqZEj02SE//M/eVdktc5hYValBgyAu++GY46BT30KLr0074qsljkszKpY62jviRPh61/3aG/LjpdV\nNatyvXrB1KlptPd//Eca7X3FFR7tbV3LYWFWA3r0gP/+bxg4EL71rTRwb8YM6NMn78qsVrgbyqxG\nSPDNb8LPfw63357ulPJob+sqDguzGnPeeXDttbBgQRqLsXx56deYleKwMKtBZ5yRpgR5/nk4/HBY\ntCjviqzaOSzMatTRR2862vvBB0u/xmxLHBZmNeygg9LgvV13TTPW3npr3hVZtXJYmNW4vfdOa3vv\nt19aE+Oqq/KuyKqRw8KsDgwcCL/7Xeqa+sQn0ngMs63hsDCrEzvumLqhzjorjfT++tehRpazsW7g\nQXlmdaRXL5g2LY32vvTSNNr7F7/waG8rzWFhVmcaGuAnP0ldUxdfDK+8kkZ7v+c9eVdmlczdUGZ1\nSIJ/+zf42c9S19QJJ8CrXuXeOuCwMKtjn/1sGu09f34ai7FiRd4VWaVyWJjVuTPPTHNJLV+eRnsv\nXpx3RVaJHBZmxrHHwj33wNq1cMQR8Ic/5F2RVRqHhZkBMGpUGu29yy5w3HFw2215V2SVxGFhZn+3\nzz4pMEaMSKO9r74674qsUjgszGwTAwemLqkjj4RCAX70o7wrskrgsDCzd2kd7X3mmfCVr8AFF3i0\nd71zWJjZZvXuDdOnp9trf/AD+PSn03TnVp88gtvMtqihAX76U9h9d/j2t9No72uvTUFi9cVnFmbW\nISlNCzJlCtxyC1xxRd4VWR4cFmZWls99Dg48EH7zm7wrsTw4LMysbM3N8NBDsHRp3pVYd3NYmFnZ\nJk9O3VIef1F/HBZmVrbBg+GYY1JY+Fba+uKwMLOtUijA00/DvHl5V2LdyWFhZlvlzDPTinu+0F1f\nMg0LSWMlLZG0VNKFW2gzQdIiSQslTW2z/wfFfYsl/bckZVmrmZVnp53g1FPhmmtg3bq8q7HukllY\nSGoApgAnASOByZJGtmszHLgIGBMR+wNfLu4/HBgDHAgcABwKHJ1VrWa2dQoFWLUKfvvbvCux7pLl\nmcVoYGlELIuItcA1wPh2bc4FpkTEawARsbK4P4DeQCPQC9gOeDnDWs1sK5x0UprK3F1R9SPLsBgE\nLG/zeEVxX1sjgBGS7pf0kKSxABHxIHA38GJxmxMR71q/S9J5kloktaxatSqTgzCzd2tshAkT4IYb\n4M03867GukOWYbG5awztb7brCQwHjgEmA5dL2lnS+4D3A3uRAuY4SUe9680iLouIpoho6t+/f5cW\nb2YdKxTgrbfgxhvzrsS6Q5ZhsQIY3ObxXsALm2kzKyLWRcQfgSWk8DgdeCgi1kTEGuA24EMZ1mpm\nW+nww2HoUA/QqxdZhsU8YLikYZIagUnATe3a3AgcCyCpH6lbahnwHHC0pJ6StiNd3PYy8mYVpEcP\nOPtsmDsXXnop72osa5mFRUSsB84H5pC+6GdExEJJl0gaV2w2B1gtaRHpGsXXImI1cB3wDPAE8Bjw\nWETcnFWtZtY5zc2wYUO6jdZqm6JGxuw3NTVFS0tL3mWY1Z0PfjDNF+V/ftVJ0vyIaCrVziO4zWyb\nFAowfz489VTelViWHBZmtk0mTUrXL3yhu7Y5LMxsm+yxBxx/fBqgVyO92rYZDgsz22aFAvzpT/DA\nA3lXYllxWJjZNjv9dNh+e3dF1TKHhZltsx12gPHjYfp0WLs272osCw4LM+sShQK8+ircfnvelVgW\nygoLSddLOkWSw8XMNuujH4V+/TwTba0q98v/Z8DZwNOSvi9pvwxrMrMqtN126Tbam2+GP/8572qs\nq5UVFhHx24hoBg4B/gTMlfSApHOKczeZmdHcDG+/DTNn5l2JdbWyu5Uk7QZ8CvhHYAHwX6TwmJtJ\nZWZWdQ47DPbZx11RtajcaxYzgfuA9wCnRsS4iJgeEV8A+mZZoJlVDyld6L77bnj++byrsa5U7pnF\nTyJiZET834h4se0T5UxAZWb1o7k5jeSeNi3vSqwrlRsW75e0c+sDSbtI+lxGNZlZFRs+HEaPdldU\nrSk3LM6NiNdbH0TEa8C52ZRkZtWuUIDHHoMnn8y7Eusq5YZFD0l/X1NbUgPQmE1JZlbtJk6EhgZP\n/1FLyg2LOcAMScdLOg6YBnicpplt1oABcOKJKSw2bMi7GusK5YbFBcBdwD8DnwfuBL6eVVFmVv2a\nm2H5crjvvrwrsa7Qs5xGEbGBNIr7Z9mWY2a1Yvx46NMnXeg++ui8q7FtVe44i+GSrpO0SNKy1i3r\n4sysevXpA2ecAddem0Z1W3Urtxvql6SzivXAscCvgauyKsrMakOhkOaJuvXWvCuxbVVuWGwfEXcC\niohnI+LbwHHZlWVmteC442DgQI+5qAXlhsXbxenJn5Z0vqTTgQEZ1mVmNaBnT5g8GWbPhtdey7sa\n2xblhsWXSfNCfRH4IFAAPplVUWZWOwqFtHreddflXYlti5JhURyANyEi1kTEiog4JyLOjIiHuqE+\nM6tyhxwC++3nrqhqVzIsIuId4INtR3CbmZVLSmMu7r0Xnn0272qss8rthloAzJL0D5LOaN2yLMzM\nasfZZ6efU6fmW4d1XrlhsSuwmnQH1KnF7WNZFWVmtWXvvWHMmNQVFZF3NdYZ5Y7gPifrQsysthUK\n8M//nGajPfjgvKuxrVVWWEj6JfCu/x+IiE93eUVmVpPOOgu+8IV0duGwqD7ldkPdAswubncCOwJr\nsirKzGrPbrvBySen6xbvvJN3Nba1ygqLiLi+zXY1MAE4INvSzKzWFArw4ovwu9/lXYltrXLPLNob\nDgzpykLMrPZ97GOw444ec1GNyp119k1Jb7RuwM2kNS7MzMq2/fZw5plw/fXw17/mXY1tjXK7oXaI\niB3bbCMi4vpSr5M0VtISSUslXbiFNhOKU58vlDS1zf4hku6QtLj4/NByD8rMKlehAG++CTffnHcl\ntjXKPbM4XdJObR7vLOm0Eq9pAKYAJwEjgcmSRrZrMxy4CBgTEfuT5qBq9Wvg0oh4PzAaWFlOrWZW\n2Y4+GgYN8vrc1abcaxYXR8SfWx9ExOvAxSVeMxpYGhHLImItcA0wvl2bc4EpEfFa8X1XAhRDpWdE\nzC3uXxMRPmk1qwENDWlE9223wSuv5F2NlavcsNhcu1JjNAYBy9s8XlHc19YIYISk+yU9JGlsm/2v\nS5opaYGkS4tnKmZWA5qbYf16mDEj70qsXOWGRYukH0raR9Lekn4EzC/xms1NPNh+YF9P0p1VxwCT\ngcsl7VzcfyTwVeBQYG/gU+/6AOk8SS2SWlatWlXmoZhZ3g48EA44wHdFVZNyw+ILwFpgOjADeAv4\nfInXrAAGt3m8F/DCZtrMioh1EfFHYAkpPFYAC4pdWOuBG4FD2n9ARFwWEU0R0dS/f/8yD8XM8ial\nC90PPgjLluVdjZWj3Luh/hIRF7Z+MUfENyLiLyVeNg8YLmmYpEZgEnBTuzY3ktb0RlI/UvfTsuJr\nd5HUmgDHAYvKOyQzqwatM9H6Qnd1KPduqLnF7qHWx7tImtPRa4pnBOcDc4DFwIyIWCjpEknjis3m\nAKslLQLuBr4WEauLa2h8FbhT0hOkLq1fbO3BmVnlGjw43RnlmWirg6KMPyVJCyJiVKl9eWpqaoqW\nlpa8yzCzrXD55XDuufDww3DooXlXU58kzY+IplLtyr1msUHS36f3KA6Q8/8LmNk2+fjHobHRXVHV\noNyw+Ffg95KuknQVcA9pMJ2ZWaftvDOceipMm5ZupbXKVe4F7tuBJtLdStOB/0O6I8rMbJs0N8PK\nlfDb3+ZdiXWk3MWP/hH4Eun210eBDwEPku5SMjPrtJNPTmcYv/kNjB1bur3lo9xuqC+RBsc9GxHH\nAqMAj4Izs23WqxdMmAA33ABrvKRaxSo3LN6OiLcBJPWKiKeAfbMry8zqSaGQpiyfNSvvSmxLyg2L\nFcVxFjcCcyXN4t2jsc3MOmXMGBgyxNN/VLKyrllExOnFX78t6W5gJ+D2zKoys7rSo0e60P3v/w4v\nvwwDB+ZdkbW31cuqRsQ9EXFTcdpxM7MuUSjAhg0wfXreldjmdHYNbjOzLjVyJIwa5a6oSuWwMLOK\n0dwM8+bBkiV5V2LtOSzMrGJMnpymL/f0H5XHYWFmFWPPPeH441NYeCbayuKwMLOKUiikBZEeeijv\nSqwth4WZVZTTT4fevX2hu9I4LMysouy4I4wfn26hXbcu72qslcPCzCpOoQCrV8OcDtfjtO7ksDCz\ninPiibDbbu6KqiQOCzOrONttBxMnpokF33gj72oMHBZmVqEKBXj77TR1ueXPYWFmFelDH4K993ZX\nVKVwWJhZRZLS2cWdd8ILXhAhdw4LM6tYzc1pJPe0aXlXYg4LM6tYI0bAoYd6rqhK4LAws4pWKMCC\nBbBwYd6V1DeHhZlVtIkToaHBZxd5c1iYWUUbOBA+8pEUFhs25F1N/XJYmFnFKxTguefg/vvzrqR+\nOSzMrOKddhr06eMxF3lyWJhZxevTJ01dPmMG/O1veVdTnxwWZlYVmpvh9dfh1lvzrqQ+OSzMrCqc\ncAIMGOC7ovLisDCzqtCzJ0yeDDffnM4wrHs5LMysajQ3w9q1cN11eVdSfxwWZlY1mprSFCC+K6r7\nZRoWksZKWiJpqaQLt9BmgqRFkhZKmtruuR0lPS/pJ1nWaWbVoXUm2nvuSeMurPtkFhaSGoApwEnA\nSGCypJHt2gwHLgLGRMT+wJfbvc13gXuyqtHMqk9zc/rpmWi7V5ZnFqOBpRGxLCLWAtcA49u1OReY\nEhGvAUTEytYnJH0QGAjckWGNZlZl9t4bPvxhuOqqNH25dY8sw2IQsLzN4xXFfW2NAEZIul/SQ5LG\nAkjqAfwH8LUM6zOzKlUopFloH38870rqR5Zhoc3sa///AT2B4cAxwGTgckk7A58Dbo2I5XRA0nmS\nWiS1rFq1qgtKNrNqMGFCupXWYy66T5ZhsQIY3ObxXkD7xRFXALMiYl1E/BFYQgqPDwPnS/oT8P+A\nT0j6fvsPiIjLIqIpIpr69++fxTGYWQXq1w9OOgmmToV33sm7mvqQZVjMA4ZLGiapEZgE3NSuzY3A\nsQCS+pG6pZZFRHNEDImIocBXgV9HxGbvpjKz+tTcDM8/n+6MsuxlFhYRsR44H5gDLAZmRMRCSZdI\nGldsNgdYLWkRcDfwtYhYnVVNZlY7Tj0VdtjBYy66i6JGbidoamqKlpaWvMsws250zjkwcya89BJs\nv33e1VQnSfMjoqlUO4/gNrOqVSjAG2/ALbfkXUntc1iYWdU65hjYYw93RXUHh4WZVa2GBjj77LTG\nxWpf7cyUw8LMqlqhAOvXw7XX5l1JbXNYmFlVO+gg2H9/d0VlzWFhZlVNSmMu7r8fli3Lu5ra5bAw\ns6p39tnp59SpHbezznNYmFnVe+974aij0lxRNTJ0rOI4LMysJhQK8NRT8MgjeVdSmxwWZlYTPv5x\naGz0he6sOCzMrCbssgucckpaQW/9+ryrqT0OCzOrGYUCvPwy3HVX3pXUHoeFmdWMk0+GnXd2V1QW\nHBZmVjN6907XLmbOhL/8Je9qaovDwsxqSqGQgmLWrLwrqS0OCzOrKUceCYMHe33uruawMLOa0qNH\nmv5jzhxYuTLvarK1Zg389Kfwwx9m/1kOCzOrOc3N8M47MH163pVkY+lS+Jd/gUGD4POfh9tuy37k\nusPCzGrOAQek2WhrqSsqAu64I609PmIE/OQnaVzJgw+m/VK2n++wMLOaVCjAH/4ATz+ddyXbprWr\naeRIOPFEePhh+Na34Nln08SJH/pQ9kEBDgszq1GTJ6cv0Wo9u2jf1dS3L1x1FTz3HHznO7Dnnt1b\nj8PCzGrSoEFw7LFpgF61zETbUVfTww+ns6VevfKpzWFhZjWrUIBnnklftJWsUrqaOuKwMLOadcYZ\naVR3pU7/UWldTR1xWJhZzdppJxg3Dq65Btaty7uapJK7mjrisDCzmtbcDK+8kr6g81QNXU0d6Zl3\nAWZmWRo7FnbdNd0Vdcop3f/5S5fClClw5ZXwxhvQ1JS6ms46qzLPILbEYWFmNa2xESZOhF/9Ct58\nE3bYIfvPjIC5c+HHP4bZs6GhIYXDF78Ihx1W2WcQW+JuKDOreYUCvPUW3HBDtp9T7V1NHXFYmFnN\n+/CHYdiw7O6KeuaZ6rmrqbMcFmZW86R0ofvOO+HFF7vmPdve1TR8ePXc1dRZDgszqwvNzbBhQ7qN\ndlvUcldTRxwWZlYX9tsv3YnU2a6oeuhq6ojDwszqRnMzPPIILF5cXvt662rqiMPCzOrGpElpJb1S\nM9HWa1dTRzINC0ljJS2RtFTShVtoM0HSIkkLJU0t7jtY0oPFfY9LmphlnWZWH3bfHT7ykRQWGza8\n+/l672rqSGaD8iQ1AFOAjwArgHmSboqIRW3aDAcuAsZExGuSBhSf+ivwiYh4WtKewHxJcyLi9azq\nNbP6UCjAP/wDPPAAHHFEbQ6gy0KWI7hHA0sjYhmApGuA8cCiNm3OBaZExGsAEbGy+PN/WxtExAuS\nVgL9AYeFmW2T006D97wHLrsMHn88hcRTT8GAAamr6Z/+qb7PILYky7AYBCxv83gFcFi7NiMAJN0P\nNADfjojb2zaQNBpoBJ7JrlQzqxd9+6bAuOqqtFXrXE3dLcuw2NzJW/v1qnoCw4FjgL2A+yQd0Nrd\nJGkP4CrgkxHxrh5GSecB5wEMGTKk6yo3s5r2zW+myQWbm93VVK4sL3CvAAa3ebwX8MJm2syKiHUR\n8UdgCSk8kLQjMBv4ZkQ8tLkPiIjLIqIpIpr69+/f5QdgZrXp/e9P3U/1eFdTZ2UZFvOA4ZKGSWoE\nJgE3tWtzI3AsgKR+pG6pZcX2NwC/johrM6zRzMzKkFlYRMR64HxgDrAYmBERCyVdImlcsdkcYLWk\nRcDdwNciYjUwATgK+JSkR4vbwVnVamZmHVNE+8sI1ampqSlaWlryLsPMrKpImh8RTaXaeQS3mZmV\n5LAwM7OSHBZmZlaSw8LMzEpyWJiZWUk1czeUpFXAs9vwFv2AV7qonDzVynGAj6VS1cqx1MpxwLYd\ny3sjouSo5poJi20lqaWc28cqXa0cB/hYKlWtHEutHAd0z7G4G8rMzEpyWJiZWUkOi40uy7uALlIr\nxwE+lkpVK8dSK8cB3XAsvmZhZmYl+czCzMxKclgUSfqupMeLM9zeUVz7uypJulTSU8XjuUHSznnX\n1FmSzpK0UNIGSVV354qksZKWSFoq6cK869kWkq6UtFLSk3nXsi0kDZZ0t6TFxb9bX8q7ps6S1FvS\nw5IeKx7LdzL7LHdDJZJ2jIg3ir9/ERgZEZ/NuaxOkfRR4K6IWC/p3wEi4oKcy+oUSe8HNgA/B74a\nEVUztbCkBuB/gY+QFvqaB0yOiEUdvrBCSToKWENaZ+aAvOvprOIKnHtExCOSdgDmA6dV45+LJAF9\nImKNpO2A3wNf2tKCcdvCZxZFrUFR1Id3LwFbNSLijuJ6IgAPkVYprEoRsTgiluRdRyeNBpZGxLKI\nWAtcA4zPuaZOi4h7gVfzrmNbRcSLEfFI8fc3SevtDMq3qs6JZE3x4XbFLZPvLodFG5K+J2k50Az8\nW971dJFPA7flXUSdGgQsb/N4BVX6pVSrJA0FRgF/yLeSzpPUIOlRYCUwNyIyOZa6CgtJv5X05Ga2\n8QAR8a8RMRi4mrTKX8UqdSzFNv8KrCcdT8Uq51iq1OZWd67aM9ZaI6kvcD3w5XY9C1UlIt6JiINJ\nPQijJWXSRdgzizetVBFxQplNpwKzgYszLGeblDoWSZ8EPgYcHxV+YWor/lyqzQpgcJvHewEv5FSL\ntVHs378euDoiZuZdT1eIiNcl/Q4YC3T5TQh1dWbREUnD2zwcBzyVVy3bStJY4AJgXET8Ne966tg8\nYLikYZIagUnATTnXVPeKF4WvABZHxA/zrmdbSOrferejpO2BE8jou8t3QxVJuh7Yl3TnzbPAZyPi\n+Xyr6hxJS4FewOriroeq+M6u04EfA/2B14FHI+LEfKsqn6STgf8EGoArI+J7OZfUaZKmAceQZjh9\nGbg4Iq7ItahOkHQEcB/wBOnfO8A3IuLW/KrqHEkHAv9D+vvVA5gREZdk8lkOCzMzK8XdUGZmVpLD\nwszMSnJYmJlZSQ4LMzMryWFhZmYlOSzMtoKkNaVbdfj66yTtXfy9r6SfS3qmOGPovZIOk9RY/L2u\nBs1aZXNYmHUTSfsDDRGxrLjrctLEfMMjYn/gU0C/4qSDdwITcynUbDMcFmadoOTS4hxWT0iaWNzf\nQ9JPi2cKt0i6VdLHiy9rBmYV2+0DHAZ8MyI2ABRnp51dbHtjsb1ZRfBprlnnnAEcDBxEGtE8T9K9\nwBhgKPABYABp+usri68ZA0wr/r4/aTT6O1t4/yeBQzOp3KwTfGZh1jlHANOKM36+DNxD+nI/Arg2\nIjZExEvA3W1eswewqpw3L4bI2uLiPGa5c1iYdc7mph/vaD/AW0Dv4u8LgYMkdfRvsBfwdidqM+ty\nDguzzrkXmFhceKY/cBTwMGlZyzOL1y4Gkibea7UYeB9ARDwDtADfKc6CiqThrWt4SNoNWBUR67rr\ngMw64rAw65wbgMeBx4C7gK8Xu52uJ61j8SRp3fA/AH8uvmY2m4bHPwK7A0slPQH8go3rXRwLVN0s\nqFa7POusWReT1Dci1hTPDh4GxkTES8X1Bu4uPt7She3W95gJXFTF649bjfHdUGZd75bigjSNwHeL\nZxxExFuSLiatw/3cll5cXCjpRgeFVRKfWZiZWUm+ZmFmZiU5LMzMrCSHhZmZleSwMDOzkhwWZmZW\nksPCzMyFbalAAAAAB0lEQVRK+v/1R8gIHtWkRAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x47f66518>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "C_s =np.logspace(-3, 3, 7)\n",
    "accuracy_s = []\n",
    "for i, oneC in enumerate(C_s):\n",
    "    tmp = fit_grid_point_Linear(oneC, X_train, y_train, X_val, y_val)\n",
    "    accuracy_s.append(tmp)\n",
    "x_axis = np.log10(C_s)\n",
    "plt.plot(x_axis, np.array(accuracy_s), 'b-')\n",
    "plt.legend()\n",
    "plt.xlabel('log(C)')\n",
    "plt.ylabel('accuracy')\n",
    "plt.savefig('SVM_RentListing.png')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 运行时间15mins"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### RBF核SVM正则参数调优"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "RBF核是SVM最常用的核函数。需要调整的正则超参数包括C和核函数的宽度gamma，C 越小，决策边界越平滑；gamma越小决策边界越平滑。\n",
    "这里采用交叉验证"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def fit_grid_point_RBF(C, gamma, X_train, y_train, X_val, y_val):\n",
    "    SVC3 = SVC(C=C, kernel='rbf', gamma=gamma)\n",
    "    SVC3 = SVC3.fit(X_train, y_train)\n",
    "    \n",
    "    accuracy = SVC3.score(X_val, y_val)\n",
    "    \n",
    "    print(\"accuracy: {}\".format(accuracy))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 0.6987134028973762\n",
      "accuracy: 0.6991186303312734\n"
     ]
    }
   ],
   "source": [
    "C_s = np.logspace(-1, 2, 4)\n",
    "gamma_s = np.logspace(-5, -2, 4)\n",
    "\n",
    "accuracy_s = []\n",
    "for i, oneC in enumerate(C_s):\n",
    "    for j, gamma in enumerate(gamma_s):\n",
    "        tmp = fit_grid_point_RBF(oneC, gamma, X_train, y_train, X_val, y_val)\n",
    "        accuracy_s.append(tmp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "accuracy_sracy_sracy_sraccuracy_sracy_ssracy_s1 = np.array(accuracy_s).reshape(len(C_s), len(gamma_s))\n",
    "x_axis = np.log10(C_s)\n",
    "for j, gamma in enumerate(gamma_s):\n",
    "    plt.plot(x_axis, np.array(accuracy_s1[:, j], label = \"Test - log(gamma) - \" + gamma))\n",
    "x_axis = np.log10(C_s)\n",
    "\n",
    "plt.legend()\n",
    "plt.xlabel('log(C)')\n",
    "plt.ylabel('accuracy')\n",
    "plt.savefig('SVM_RBF_RentListing.png')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
